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ABSTRACT 

This report describes a project undertaken at the 
University of Tennessee, Knoxville (UTK) between January 1, 1982 and 
March 31, 1984, which was designed to increase the use in program 
assessment and in program improvement of information derived from 
measures of; (1) student achievement in general education; (2) 
student achievement in the major field; and (3) student opinions 
concerning the quality of academic programs and services. After a 
brief program description, the origins of the UTK project are 
examined, including proposal development, the student outcome data 
available, and goals of the campus project* Project operations are 
then reported including the following topics: project structure and 
organization; project activities; campus involvement strategies; data 
presentation strategies; and project continuation. Project impact 
included: changes in curriculum and instruction (e.g., participation 
in one or more evaluative procedures was made mandatory for 
students); changes in student services (e.g., improved advising 
services) ; and changes in institutional planning and evaluation 
activities (e.g., increased use of student outcome information in 
ma^or field program evaluation) . The major portion of the document 
consists of 12 appendixes which provide details of the program's 
activities and findings. (DB) 
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Program Description 
The NCHEMS/Kellogg Student Outcomes Project undertaken at the Univer- 
sity of Tennessee, Knoxvllle (UTK) between J<-nuary 1, 1982 and March 31, 
1984 was designed to Increase the use in program assessment and Improvement 
of information derived from measures of (a) student achievement in general 
education, (b) student achievement in the major field, and (c) student opin- 
ion concerning the quality of academic programs and services • The project 
was initiated to address a generally recognized need to Integrate and make 
more use of information derived from tests and surveys administered in con- 
nection with (a) students' entry to the University, (b) accreditation self 
studies and peer reviews, and (c) the Institution's own comprehensive aca- 
demic program reviews ♦ Recognition of this need was intensified by the 
necessity to respond to the instructional evaluation (or "performance fund- 
ing") requirements established in 1979 for all the state's publicly sup- 
ported institutions of higher educatiori by the Tennessee Higher Education 
Commission (THEC). Under the Instructional evaluation program, up to five 
percent of an institution's annual state budgetary allocation is to be 
awarded on the basis of evidence demonstrating the accomplishment of the fol- 
lowing performance criteria (See Appendix 1 for the complete schedule of cri- 
teria) : 

a) calculating the percentage of programs eligible for accreditation 
that is accredited. 

b) calculating the percentage of programs which, within a five year 
period, has undergone peer review and/or administered to 

majors a comprehensive field exam. Maximum credit for this stan- 
dard is awarded if student performance on the field exam improves 
over time or exceeds the performance of students in similar pro- 
grams at comparable institutions* 

c) measuring value added via the general education component of the 
curriculum using the American College Testing College Outcome 
Measures Project exam (ACT COMP) and demonstrating that perfor- 
mance of seniors exceeds the mean of value added computed for 
seniors at a group of comparable institutions* 

d) conducting surveys of enrolled students, alumni, community members, 
and/or employers and demonstrating that generalizations about the 
quality of academic programs or services derived from the surveys 
have formed the bases for specific improvements in campus programs/ 
services* O 
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e) implementing a campus-wid»i plan for instructional improvement based 
on information derived from procedures 1-^ above, as well as other 
sources ♦ 

The University of Tennessee, Knoxville is a land-grant institution, and 
Tennessee's major comprehensive university* Its mission encompasses instruc- 
tion, research, and public service* In Fall 1983 approximately 26,500 stu- 
dents (20,500 undergraduates and 6000 graduate students) were enrolled in 
one of the institution's 170 undergraduate or 168 graduate programs* The 
size of the student body and faculty, and the diversity of academic programs 
offered, make UTK an appropriate environment for a large-scale demonstration 
of the application of student outcome information in comprehensive program 
evaluation in higher education. 

Origins of the UTK Project 

Proposal Development 

The opportunity to respond in the summer of 1981 to a request for pro- 
posals from NCHEMS and the Kellogg Foundation on the topic of increasing the 
use of student outcome information provided a focal point for a rather vaguely 
recognized notion that the University of Tennessee, Knoxville was collecting 
a great deal of information about students that could be used much more effi- 
ciently and effectively within the institution. For freshmen, ACT scores and 
data from the accompanying Class Profile Report had been accumulating for mox 
than a decade. Until 1981 additional information on freshmen had been col- 
lected via the Cooperative Institutional Research Program (CIRP) question- 
naire. Group means for scores achieved by seniors on licensing exams. Grad- 
uate Record Exams, and other comprehensive assessments were compiled for in- 
clusion in self-study documents prepared for accreditation reviews and the 
internal co *rehen?ive program review process. In addition, some units had 
conducted surveys of student and/or alumni opinion regarding the quality of 
academic programs and services. A two-year study of general education ini- 
tiated by Chancellor Jack Reese in 1979 had prompted the faculties of most 
colleges to undertake student surveys and analyses of student records that 
produced a body of information on stuaent curricular choices within the Uni- 
versity. Prior to 1981 data from these disparate sources were stored in in- 
dividual files across the campus; no office had been charged with the respon- 
sibility of attempting to integrate the information and use it in an 
institution-wide comprehensive program evaluation process. 
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At the time that the NCHEMS-Kellogg RFP arrived two other developments 
were pushing campus administrators toward formal acknowledgement of the need 
to develop a more systematic approach to collection and use of outcome infor- 
mation at UTK» In Fall 1979 the Tennessee Higher Education Commission intro- 
duced an Instructional Evaluation Schedule as a supplement to its enrollment- 
based formula for allocating state funds among institutions. The Schedule 
contained the outline of an institution-wide approach to gathering evidence 
of student achievement in general education and in the major field, and of 
student opinion concerning academic programs and services. In addition, the 
Executive Vice Chancellor for Business, Planning and Finance, Homer Fisher, 
undertook a strategic planning initiative in 1981 that placed new emphasis 
on the assessment of outcomes in setting institutional priorities and making 
decisions about the allocation of resources. 

The NCHEMS-Kellogg project was viewed by Chancellor Reese and his staff 
as a vehicle for organizing a careful study of the kinds and the quality of 
student information available to the campus and making recommendations for 
improving its quality and increasing its use in planning and decision-making. 
The chief advocate for involving the University in the project was Trudy 
Banta, a professor from the College of Education who was participating in 
an administrative internship in the Office of the Chancellor in 1981. 
Banta 's background in educational measurement and program evaluation prompted 
her interest in providing leadership for an effort to integrate student out- 
come information in comprehensive institutional program evaluation. 

The kinds of student information described in the foregoing paragraphs 
were stored in the offices of deans across the campus, in the Office of 
Admissions and Records in the Division of Student Affairs, and in the Office 
of Institutional Research in the Division of Business, Planning and Finance. 
The new responsibility for coordinating the response to the THEC Instruc- 
tional Evaluation Schedule had been assigned to yet another potential reposi- 
tory for data: The Learning Research Center, an independent academic unit 
responding directly to the Provost, charged with evaluating and improving 
curriculum and instruction on the campus . The decision was made to relocate 
Professor Banta upon conclusion of her internship in the Chancellor's Office 
in the Learning Research Center for the purpose of conducting the NCHEMS- 
Kellogg project. Assigning the project to Provost George \flieeler strengthened 
its linkage with the academic program review process — also conducted by the 
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Office of the Provost — and with strategic planning carried out by the Plan- 
ning and Judgeting Coordinating Committee co-chaired by Wheeler and Fisher. 

The Project proposal was written by Trudy Banta, then reviewed by mem- 
bers of the Chancellor's staff, selected deans, and the Director of the Learn- 
ing Research Center. All of these individuals expressed strong support, and 
letters of endorsement for the proposal were received from: 

(a) Chancellor Jack Reese; 

(b) The Vice Chancellors for Student Affairs and for Business, 
Planning and Finance; 

(c) The Associate Vice President for Academic Affairs; 

(d) The Dean of the Graduate School; 

(e) The Dean of the College of Business; and 

(f) The Director of the Learning Research Center. 
Student Outcome Data Available 

While the principal focus of the NCIIEMS-Kellogg project was upon out- 
come data, outcomes can be measured most accurately if compared with the re- 
sults of appropriate input measures. Thus an important source of the infor- 
mation to be considered was the data on entering freshmen provided by ACT. 
Information from the CIRP instrument also was considered for the years dur- 
ing which it was available. 

The types of information required for accreditation self-studies vary 
from one discipline to another. The guidelines for the self-study to be con- 
ducted in preparation for the internal program review at UTK are presented in 
Appendix 2. Some departments included in these reports information from one 
or more of the following sources! 

(a) the ACT COMP exam in general education, which had been administered 
annually to seniors since 1980; 

(b) the Graduate Record Exam, comprehensive assessments of achieve- 
ment in the major field, and professional examinations in archi- 
tecture, education, engineering, nursing, social work, law, and 
planning; 

(c) surveys of enrolled students such as the ACT Student Opinion 
Survey administered to a UTK sample in 1980 and in 1981; 

(d) surveys of program graduates; and 

(e) surveys of employers of program graduates. 

Selected results of the study of general education undertaken in the 
College of Liberal Arts in 1979 were published by the Learning Research 
Center in 1980 in a document entitled, Teaching/Learnin?^ Issues : Curren t 
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Student and U nit Currlcular Practice In the College of Liberal Arts, Uni- 
versity of Tennessee^ Knoyville, The campus-wide study resulted m the 
development of the document "Interim Report of the Coordinating Committee 
on General Education", which appears in Appendix 3. 

The foremost concern about all of these data was their validity: 
Skeptical faculty and administrators wondered if the instruments being used 
to measure outcomes actually were measuring the outcomes on which UTK aca- 
demic programs were based. Were the data obtained by using so-called "ob- 
jective" measuring instruments — exams and surveys — providing better in- 
formation than University personnel could obtain by talking with a few 
selected students, alumni, or employers? Was the expense of systematic data 
collection justified by an increase in validity when results were compared 
with those derived from the collective perceptions of faculty? 

Data from the ACT Student Opinion Survey had been ignored almost uni- 
versally. Would results obtained from some other instrument be more suc- 
cessful in attracting the attention of decision-makers? 

Administrators receiving scores from some of the professional exams 
were concerned because their reports did not contain subscores ~ scores 
in specialized areas of the discipline ~ that would permit assessment of 
the adequacy of students* preparation in the various components of the cur- 
riculum. 

A final concern about the student outcome information available at 
UTK was the absence of direction or motivation for integration of data from 
several sources to strengthen conclusions derived from each, and to increase 
the use of these conclusions in program planning for improvement. There was 
some recognition of the need to integrate data both within each discipline ~ 
to provide a solid basis for recommending changes in curriculiim and instruc- 
tion within the program area ~ and across disciplines — to produce recom- 
mendations for strengthening institution-wide services such as registration, 
orientation, counseling, and placement. But there was no advocate in the 
central administration for the utilization of outcome information in these 
ways. Moreover, there was no force for encouraging or motivating campus 
units to collect from a variety of sources infoniiation bearing on the quality 
of their programs. 
Goals of the Campus Pro-ject 

Tlie NCHEMS-Kellogg RFP held out the possibility of providing the impetus 
for some released time for the Project Director to organize and interpret the 
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various kinds of information from and about students that were accumulating 
in otfices across the campus. Moreover, the opportunity to contact outside 
consultants was viewed as a potential source of assistance in identifying im- 
provements that might be made in data-gathering methods and procedures, axid 
in determinipg appropriate utilization strategies* 

The principal goal of the project at the outset was to develop an or- 
ganizational framework within the University for involving administrators, 
faculty, and students in every campus unit in planning directed toward in- 
creased use of student outcome information. As a practical matter, the 
domain of outcomes was restricted to the three areas that formed the core 
of the THEC Instructional Evaluation Schedule, i.e., student achievement in 
general education, student achievement in the major field, and student opin- 
ion concerning the quality of academic programs and services. 

The primary goal of the project did not change, and ultimately it was 
achieved. However, the timeframe for full achievement was a year longer 
than had been anticipated initially. Whereas the first year of the project 
had been viewed as one of action and implementation, the readiness of the 
campus for widespread involvement in outcomes utilization had been over- 
estimated. Consequently, the first year was devoted to deliberation and 
study of the feasibility and potential impact of collecting and using the 
types of outcome information specified in the THEC performance standards. 

The campus commitment to the NCHEMS-Kellogg initiative did result in re- 
leased time for the Project Director to provide a central focus for collec- 
tion and use of outcome information. The opportunity to have Aubrey Forrest, 
consultant from ACT, come to the campus tvice — once for discussions and 
once for a more formal workshop on the measurement of achievement in general 
education — constituted the most effective vehicle utilized during the 
project for involving faculty and students in the process of considering the 
use of outcomes measures. 

The original proposal to NCHEMS indicated that at an early date — dur- 
ing the 1982 Winter Quarter ~ department heads would be asked to develop, 
with assistance from faculty and students, a written statement of the unit's 
plans to collect and use outcome information. However, between the time the 
proposal was submitted and the date the award was announced, the Project 
Director met individually with each of the nine deans of colleges enrolling 
undergraduates to obtain their impressions of the quality and usefulness of 
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outcome data currently available to them. This series of interviews revealed 
considerable dissatisfaction with the quality of much of the data and some 
skepticism about its usefulness. (See Appendix 4 for a sample interview tran- 
script.) The experience indicated that a careful assessment of the adequacy 
of information-gathering methods needed to be undertaken before utilization 
strategies could be considered. 

As a direct consequence of the interviews with academic deans, the first 
project activity in January 1982 was the convening cf a Technical Advisory 
Council composed of deans' representatives ~ in most cases the associate 
deans ~ to undertake a detailed assessment of outcome information-gathering 
instruments and methods. The work initiated by this group, and the process 
of following up its recommendations, occupied two years. This interim phase 
was critical to further implementation of the primary project goal, and it 
resulted in the development of a number of important materials and strategies. 
In January 1984 the request for all department heads to develop plans for col- 
lecting and using outcome information finally was issued. 

Project Operations 
Project Structure and Organization 

During 1982 the Project Director was a member of the Chancellor's staff, 
but also reported through the Director of the Learning Research Center (LRC) 
to the Provost (then the Vice Chancellor for Academic Affairs) for purposes 
of carrying out administrative functions related to the NCHEMS-Kellogg 
project. Initially the visibility provided the project by the linkage with 
the Chancellor's Office was very helpful. But the "Kellogg Project", as it 
came to be knoiro, quickly established its own identity and continued without 
disruption when the Director's internship with the Chancellor was completed 
and she moved to the LRC. 

In 1982 the LRC Director was advised by an Instructional Evaluation 
Advisory Committee (lEAC) that included the following individuals: 

a) The Vice Chancellors for Student Affairs and Business and Finance, 

b) The Assistant Vice President for Academic Affairs (UT system admin- 
istration) , 

c) The Associate Vice Chancellors for Academic Affairs and for Planning 
and Administration, 

d) The Deans for Graduate Studies and Business Administration, 

e) Representatives of the Faculty Senate and the Student Government 
Association, and 



1?. 



0 Selected faculty with interest and expertise in measurement of 
outcomes. 

The lEAC anctioned as the oversight committee for the Kellogg Project. 

In order to accomplish the initial studies of outcome information- 
gathering strategies on the campus a Technical Advisory Council (TAG) of 
deans' representative was established under the aegis of the lEAC. At the 
first meeting of the TAG the decision was made to form three task forces^ 
one to carry out specialized investigations in each of the three targeted 
outcome areas: Achievement in general education, achievement in the major 
field, and opinion concerning the quality of programs and services. In each 
2rea faculty and graduate students with relevant expertise wer^ selected and 
asked to join the deans' representatives on the task force. 

For six months each of the task forces met at 3-4 week intervals for 
discussion and review of materials. The Project Director chaired all of the 
meetings and assigned the primary responsibility for developing the final re- 
port for each task force. The three task force reports (see Appendix 5) were 
reviewed in June 1982 by the TAG, and recommendations for action based on the 
reports were conveyed to the lEAG in a series of meetings of that group in 
June and July. 

A key recommendation of the lEAG was that at least one pilot project 
focused on utilization of outcome information be carried out in each of nine 
colleges v.ith Kellogg funds during 1982-83. Recommendations bearing on 
measurer.ent of student opinion and achievement in general education were to 
be implemented centrally by LRG staff and interested faculty. After July 
1982 meetings of the three task forces, the TAG, and the lEAG were called 
as the advice of each was needed to further the progress of the project. In 
administrative matterr the Project Director relied upon the counsel and 
assistance of the LRG Director and two Vice Provosts with responsibilities 
for instructional evaluation and academic program review. By design the 
project became as rapidly as possible an integral part of the on-going in- 
tellectual environment and administrative structure of the University of 
Tennessee, Knoxville. Gommunications about project activities flowed through 
the established channels: Ghancellor's staff (including the Provost), then 
from the Provost to the Board of Deans, and on to department heads and 
faculty. 
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Initial Project Activities 

At its first meeting the Technical Advisory Council established three 
task forces and charged each with developing a report that would: 

a) Identify one or more reliable, valid methods for measuring the 
student outcome(s) under consideration, including specifications 
for carrying out the methodology (i.e., sampling techniques, forms 
of instruments, motivation of subjects, testing conditions to be 
used) ; 

b) Describe in detail the kinds of information each method could pro- 
vide; 

c) Identify inherent weaknesses/limitations associated with each method; 

d) Estimate the cost of resources needed to implement each method; and 

e) Suggest how the information obtained could be used, in combination 
with information from other sources, in program assessment, admin- 
istrative decision-making, and planning for program improvement. 

Each task force began its deliberations by confronting the issue of 
credibility: "\Vhy should faculty and students spend time on formal tests 
and surveys designed to provide information about program quality that we 
have managed in the past to obtain informally by talking with students, 
colleagues, alumni and employers?" Independently each task force concluded 
that in facing increasing needs to demonstrate program quality, establish 
priorities for resource allocation, and justify programmatic emphases, each 
academic unit would be well seirved by having at its disposal systematically 
collected structured information from a variety of sources. 

On June 3, 1982 the Technical Advisory Council for the Kellogg project 
held its second meeting. The three task force reports were reviewed and a 
procedure was outlined for implementing the recommendations that were in- 
cluded. 

A composite rec emendation drawn from the three reports suggested that 
student outcomes informat5on be employed in each academic unit to provide 
evidence of program quality for the following purposes: 

1) in communicating with alumni and friends of the University, stu- 
dents and their parents, professional colleagues; 

2) in preparing for accreditation reviews; 

3) in strategic planning; 

4) in preparing for the comprehensive program review process; 

5) in responding to the requirements of the THEC Instructional Evalua- 
tion Schedule. 
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Student outcomes information also could be used by the department and/or 
college to suggest: 

1) changes in curriculum at the course level, i,e», course content 
updated or given new focus* 

2) changes in curriculum at the program level, i^e., courses or content 
areas added or deleted* 

3) areas for professional development of faculty. 

4) addition of, or changes in, field experiences. 

5) addition of, or changes in, seminars, colloquia, special events. 

6) changes in advising procedures. 

7) improvements in classroom or laboratory facilities and/or equipment. 

8) additions to library collections. 

9) improvements in campus-wide student services. 

Specific recommendations of the task forces included the following: 

1. The ACT COMP College Outcome Measures Project) Objective Test should 
be givan annually to all incoming freshmen and to all graduating 
seniors so that the comparisons of pre- and post-program scores may 
yield a measure of value added as a result of the educational exper- 
ience provided at UTK. 

2. Faculty in each academic unit should determine whether or not there 
is a standardized instrument available for measuring student achieve- 
ment in the major field(s) of study offered by the unit. If an accep- 
table standardized instriunent is available, the academic unit should 
require that every graduating senior take the examination. Mean 
scores for classes of students should be compared with national means 
in making judgments concerning program quality. 

If the academic unit does not have access to a standardized test of stu- 
dent achievement in the major field of study, the faculty should develop 
or adapt one or more of the following methods: 

a. evaluation by faculty of a comprehensive student achievement. 

b. evaluation of a comprehensive student achievement by an external 
reviewer. 

c. end-of-program assessment by seniors, reporting perceptions of 
their own achievement. 

d. retrospective assessment of their own achievement by alumni. 

e. assessment by employers of the competencies of alumni one or several 
years after graduation. 

3. A measure of student satisfaction with academic programs and ser- 
vices should be developed and administered at least every other 
year to representative samples of (a) enrolled students and (b) 
recent graduates at UTK. The resulting information should be 
reviewed by an appropriate body such as the University-wide In-- 
structional Evaluation Advisory Committee. Following comparison 



of current data with that obtained in previous administrations 
of the instrument, and perhaps some follow-up interviewing to in- 
crease understanding of certain responses > the oversight group 
should issue a report on the findings to deans and department 
heads, including recommendations for program improvements that 
seem warranted. 

In addition to the University-wide survey of student opinion, 
each college or department should conduct its own survey of cur- 
rently enrolled majors and non-majors at least every other year 
and of alumni at least once every four years. A group of faculty 
with interests in curriculum development, program evaluation, 
measurement, and survey design and analysis should be established 
to oversee the implementation of this survey methodology — to sug- 
gest student and course sampling procedures and to develop data 
analysis and reporting strategies. 

On June 10, 1982 the Instructional Evaluation Advisory Committee held 
the first of several meetings for the purpose of considering the task force 
reports and the recommendations of the Technical Advisory Council. The 
Kellogg Project "Plan of Work" prepared in February 1982 described an in- 
tention to utilize task force findings in preparing a detailed request for 
all department heads to submit plans for using student outcome information 
by December 1982. The lEAC decided it was too soon to approach all depart- 
ment heads with 3uch a request. Instead the decision was made to ask each 
of the nine deans of colleges enrolling undergraduates to submit one or 
more proposals for pilot projects designed to increase the use of student 
outcome Information in a department or in the college* The argument was 
advanced that the experience of a few carefully selected and monitored 
pilot efforts would be instructive for other units when a general request 
for plans vjas issued in late 1983 or early 198A. 

The lEAC also approved the TAC recommendations for administering cen- 
trally measurement programs for student and alumni opinion and student 
achievement in general education. Responsibility for those initiatives was 
vested in the Learning Research Center. 

Early consideration of the Kellogg Project by members of the Chan- 
cellor's staff and the Board of Deans ensured that the individuals whose 
cooperation was essential to the success of the project were fully informed 
about it. The TAC recommended to the lEAC that a Chancellor's Newsletter 
with campus-wide distribution be prepared early in Fall Quarter 1982 to 
acquaint faculty with the aims of the project and progress to date. The 
lEAC considered that recommendation and decided instead to move more quietly 
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and cautiously, first building a base of successful experience in data col- 
lection and utilization both centrally and in pilot units before calling 
broad attention to the effort* 
Subsequent Activities 

As soon as the Instructional Evaluation Advisory Committee had con- 
cluded its review of the Technical Advisory Council's recommendations and 
had provided clear direction for the next phase of the Kellogg Project, 
the Project Director wrote to the nine deans to thank them for naming repre- 
sentatives to the TAG and to ask that these individuals be encouraged to 
continue their work in the same areas but as members of "advisory com- 
mittees" rather than task forces. The three advisory committees would serve 
to oversee the implementation of lEAC-approved recommendations for measuring 
(a) student achievement in general education, (b) student achievement in the 
major field, and (c) student opinion of the quality of academic programs and 
services. 

Next, the Project Director scheduled interviews with each of the nine 
deans and their representative(s) to the TAC. The purpose of this series of 
meetings, which took place in August 1982, was to review the work of the task 
forces and to encourage the deans to identify candidates for the pilot project 
awards. 

. le interviews v^ith deans were extremely helpful — as they had been in 
August 1981 — in providing direction for implementing the lEAC recommenda- 
tion on pilot projects. Without exception, each dean was able to identify 
during the interview a potential colle^^^^-wide or department-based project 
idea. 

In October the Project Director sent a letter to each dean (See Appen- 
dix 6) summarizing the project ideas discussed in August and suggesting that 
enclosed guidelines and an application form be used to submit one or more 
proposals by November 5. A budget of $350 - $1500 was suggested for each 
pilot project. 

Early in the Fall Quarter the General Education Task Force, reconsti- 
tuted as the Advisory Committee for General Education, met and recommended 
thai Aubrey Forrest of the American College Testing program be invited to 
discuss with interested faculty a series of questions related to the meaning 
of ACT COMP scores attained by UTK students. NCHEMS-Kellogg consulting funds 
were utilized to bring Dr. Forrest to the campus in late October. Approximately 
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30 faculty and administrators attended the presentations, and many of those 
present expressed appreciation for the opportunity to clarify their under- 
standing of the COMP exam and its purposes* Interest was generated in hav- 
ing Dr» Forrest return to the campus in the Spring to conduct a workshop for 
a broader segment of the faculty. 

The Task Force on Student Opinion, reconstituted as an advisory committee 
recommended that the Learning Research Center enter into a contractual agree- 
ment with Ken Van Liere, a professor in the Department of Sociology, and Bill 
Lyons, a professor in the Department of Political Science, who were interested 
in designing an instrument and appropriate survey methodology for measurement 
of student opinion concerning the quality of academic programs and services • 
The campus administration agreed that a locally developed instrument was 
needed to obtain information academic administrators would consider relevant 
and put to use. Therefore, a contract for services was developed for signa- 
ture by appropriate parties, and a process was initiated that would result in 
administration of a pilot-tested instrument to a sample of 2200 enrolled stu- 
dents in April 1983. 

Both the Advisory Committees on Student Opinion and Testing in the Major 
Field met in mid-November 1982 to evaluate the proposals for pilot projects 
th.'t were submitted by the nine deans. This was not a competitive process as 
had been anticipated at the outset because each dean had selected the most 
promising proposal (s) from the unit, and the Project Director wanted repre- 
sentation from each college. The advisory groups were helpful, nevertheless, 
in suggesting how some of the pilot projects might be improved through addi- 
tional directions in the letter that would be sent to each dean to announce 
the award(s). A brief description of the projects that were proposed appears 
in Table 1. The total cost of these activities was $20,500. Approximately 
$9000 in Kellogg funds were committed for the pilot projects. The University 
administration added $11,500 to the Learning Research Center budget to com- 
plete the funding package. 

The fact that all proposals for pilot projects involved the collection 
of new data attested to the fact that most UTK administrators either had no 
student outcome data or were not satisfied with the quality of that which 
was available. Despite the apparent emphasis on data collection, however, 
the proposal guidelines and subsequent communications with directors of the 
projects emphasized the need for utilization of the resulting information in 
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Table 1. Description of UTK Kellogg Pilot Projects 



Unit Proposing the Project 
Agriculture 

Architecture 
Business 



Communications 
Education 

Engineering 
Home Economics 



Liberal Arts 



Biology 

Botany 

English 

History 

Microbiology 

Psychology 

Zoology 

Geography 



Nursing 



Description of Project 

Locally developed com- 
prehensive exam 

Survey of graduates 

Survey of enrolled stu- 
dents using Tellus 
technology that permits 
self-report and instan- 
taneous tabulation of 
responses 

Survey of graduates 

Surveys of enrolled stu- 
dents and graduates in 
Recreation programs 

Survey of enrolled stu- 
dents 

Interviews with employers 
of graduates of Textiles, 
Merchandising, and Design 
programs 



Graduate Record Exams 
for seniors 



Amount of Project Award 
$1500 

700 
500 



Locally developed com- 
prehensive exam 

Mosby AssessTest for 
seniors 



700 
500 

350 
550 



12,700 



1,500 



1,500 



$20,500 

combination with that from other sources to make decisions and take action 
designed to improve programs and services. Award letters were sent to deans 
and the appropriate department heads in December and projects were initiated 
January 1, 1983. 

In order to ensure that representative samples of students actually would 
take the tests in general education and in the major field that had been pro- 
posed by the Kellogg task forces, it was necessary to add to the University 
catalog a requirement that students participate in one or more evaluative 
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procedures prior to graduation. Drafting a statement that met with approval 
of academics as well as the University's legal counsel; taking this statement 
to committees, the Undergraduate Council, and eventually the Faculty Senate; 
and gaining the faculty support necessary to pass the proposed requirement at 
each of these levels of the governance structure took much of the Project 
Director's time and energy during November, December, and January, The vote 
on the requirement (see Appendix 7) in the Faculty Senate on January 18, 1983 
was affirmative. Subsequently all candidates for graduation were informed 
that they must take the COMP exam in general education if selected to do so, 
OR take a comprehensive exam in their major field, OR complete the Student 
Satisfaction Survey (designed by Van Liere and Lyons) at the Career Planning 
and Placement Center, 

In February at the NCHEMS-Kellogg Project Director's Meeting in Boulder, 
the Project Director developed an idea that resulted in the addition of a 
significant piece of work to the scope of the UTK project. Interaction with 
otb^r project directors in Boulder led to the recognition that thw unique 
performance funding initiative in Tennessee was of interest to higher educa- 
tion personnel throughout the country: It seemed appropriate to chronicle 
the University of Tennessee's assessment of and response to the IHEC Instruc- 
tional Evaluation Schedule as these processes were assisted by the Kellogg 
Project. Back on the campus in Knoxville, a month was spent in identifying 
a dozen faculty and administrators who were willing to serve as aithors, and 
in mid-March the first of several multi-hour sessions was held for the pur- 
pose of developing a monograph entitled. Performance Funding in Higho.r Educa- 
tion; A Critical Analysis , 

By mid-March 1983 the Kellogg Project at UTK was well positioned to 
achieve the results envisioned by the lEAC in July 1982, Three advisory com- 
mittees were providing direction for initiatives to be undertaken by the LRC 
with the support of the central administration, the pilot projects were pro- 
ceeding with minimal needs for leadership from anyone outside the units con- 
ducting the projects, and the monograph team had embarked on its assignments 
The stage was set for accomplishing the results described in succeeding 
paragraphs. 

In March a sample of seniors representative of the range of grade point 
averages within each of the nine undergraduate colleges was selected to take 
the ACT COMP exam on May 7. Each dean sent personal letters to the seniors 
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selected to represent his or her college. Hearings were scheduled to review 
individuals' concerns about the testing dace, and an alternative date was 
established for those seniors who could not come for testing on May 7* By 
the end of May approximately 700 seniors had taken the test. 

Seven Liberal Arts department heads contacted seniors enrolled in their 
programs and asked them to take the Graduate Record Advanced Test in their 
field. On April 15, 277 seniors took the Graduate Record Exam at UTK. 

In April Kent Van Liere and Bill Lyons administered the Student Satis- 
faction Survey to a random sample stratified by college of 2200 enrolled 
students. In addition, samples were drawn from the students in five selected 
departments (Human Services; Marketing & Transportation; Biology; Textiles, 
Merchandising & Design; and Political Science) in order to provide informa- 
tion at the departmental as well as college and university-wide levels. More 
than 70 percent of the students surveyed returned completed questionnaires. 

On May 10 Aubrey Forrest visited the campus again, this time to conduct 
a workshop for faculty on the measurement of achievement in general education 
using the ACT COMP exam. Nine deans were asked to identify a designated num- 
ber (from 3 for the smallest colleges to 15 for Liberal Arts) of faculty with 
interests in general education. A letter was sent from the Director of the 
Learning Research Center inviting each of the faculty members identified by 
the deans to attend one of two identical workshop sessions — - one scheduled 
for 9 a.m. to 12 noon, the other from 1:30 to 4:30 p.m. At noon a luncheon 
was held at the Faculty Club for participants in both morning and afternoon 
sessions. The officers of the Student Government Association also were in- 
vited to attend a workshop session, and four of them did so, including the 
President and Vice President. Discussion generated during the workshop in- 
dicated a high level of interest in Dr. Forrest's presentation. 
Campus Involvement Strategies 

The Kellogg Project provided the impetus for establishing in the Learn- 
ing Research Center a central focus for collection, analysis, and use of 
credible outcome information at UTK. Through a series of careful steps under- 
taken over a period of two years this new initiative was achieved through 
existing administrative structures and channels of communication, i.e.. 
Chancellor's staff Board of Deans (chaired by Provost) -> Department 
Heads — > Faculty. When appropriate, faculty were contacted directly through 
the Faculty Senate, limited numbers of faculty and students were invited to 
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participate in the work of the task forces and the advisory committees, 
larger numbers took part in the sessions conducted by Aubrey Forrest, 

Since the measurement of general education was conducted centrally, it 
was very helpful to be able to ')ring in an outside consultant to further 
understanding of the process and to heighten interest in potential uses of 
data from the COMP exam. In the arees of testing in the major field and 
assessing opinion of students and graduates, department heads, faculty, and 
students became directly involved in determining their own objectives, col- 
lecting data, interpreting the findings, and using results to improve their 
programs. In the University-wide survey of student opinion Professors Van 
Liere and Lyons utilized interviews with students and with department heads 
as they developed questions for the sections of their instrument dealing 
with quality of programs/services in the major and of classroom experiences 
in the department. The Vice Chancellor for Student Affairs and his staff 
were directly involved in the design of the sectiions of the questionnaire 
that dealt with general university programs and services such as the li- 
brary, the counseling center, campus film series, computer services, and 
University bookstore. These involvement strategies were undertaken very 
deliberately in order to give potential users of outcome information a 
vested interest in obtained results. 

The Kellogg Project Director's own management style is characterized 
by a great deal of personal contact. Telephone or face-to-face conmiunica- 
tions almost always piecede written communications. Memos contain confirm- 
ing information that provides background for suggesting directions for 
future action. Following individual contacts, often group meetings are 
scheduled for the purpose of sharing experiences and obtaining consensus 
concerning the most expeditious ways of accomplishing the next steps. As 
often as possible, personal communications with deans and department heads 
begin with a sharing of some new information generated by one or more of 
the project activities; this is followed by the inevitable request for 
additional cooperation and assistance. 

While the Kellogg Project at UTK had an identity and life of its own, 
it was inextricably associated with the THEC performance funding initiative, 
and drew much of its importance in the eyes of Tniversity administrators from 
that association. The Kellogg Project enabled UTK to view the Instructional 
Evaluation Schedule with some detachment, to approach it intellectually, to 
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scrutinize its feasibility, and to find ways of making it work for the insti- 
tution, i.e., to complement on-going processes of program evaluation and pro- 
vide information for use in making decisions about the allocation of internal 
resources. 

Data Presentation Strategies 

ACT provides ample opportunity for institutions to add their own items 
to the COMP exam and to have the responses to these items tabulated along 
with the information ACT itself collects. Prior to the May 1983 senior 
testing date at UTK, the General Education Advisory Committee assisted the 
Kellogg Project Director in developing a number of supplementary items that 
would produce information on patterns of course selection to fulfill general 
education requirements within each college. Other items were added to pro- 
vide demographic data and information about participation in a variety of 
special activities such as internships, honors courses, interdisciplinary 
studies, and student professional organizations. It was hoped that analyses 
of the effects of special experiences and patterns of course work on COMP 
exam scores would yield information that could be used in making recommenda- 
tions for improvements in curriculum and instruction* Comprehensive analy- 
ses of these relationships will take years to complete, but some preliminary 
findings were available by September 1983. 

At approximately the same time, a report was completed by Van Liere and 
Lyons on the analysis of the Student Satisfaction Survey conducted in April 
(see Appendix 8). A summary report (see Appendix 9 for a sample) combining 
information from the following sources was compiled for each college: 

a) ACT COMP exam scores and preliminary analyses of supplementary 
items; 

b) the Student Satisfaction Survey; 

c) testing in major fields; 

d) surveys of graduates, employers, or enrolled students conducted 
by the college or by one or more of its units; and 

e) student evaluations of courses initiated by individual instructors. 
At the beginning of the Fall Quarter this report x^as made available to 

each dean, and the Kellogg Project Director and the Director of the Learning 
Research Center offered to attend a faculty meeting to explain the contents. 
Seven of nine deans availed themselves of the opportunity for such a presen- 
tation. 
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The summary reports and presentations to faculties had not been con- 
ceptualized as vehicles for dissemination at the beginning of the Kellogg 
Project. Nevertheless, they were extremely effective in attracting the 
attention of faculty to the importance of student outcome information and 
in establishing the Learning Research Center as a dependable source of cred- 
ible information presented in comprehensible, usable terms. 

Chancellor Jack Reese received a private briefing on the information 
that had been compiled for the college reports. He asked if Professors Van 
Liere and Lyons could conduct further analyses of their data to shed some 
light on factors related to student retention. Subsequently information was 
obtained concerning the identity of Spring 1983 survey respondents who had 
returned to UTK in Fall 1983 and those who had not. Regression analyses re- 
vealed that overall satisfaction with the University experience was the single 
most important determinant of student retention (see report in Appendix 10). 
Implications were drawn for increasing academic and social satisfaction in 
each of nine colleges. The results of this special analysis for the Chan- 
cellor were presented first in written form to the Chancellor's staff, then 
orally to the Provost's personal staff, and orally to the Board of Deans. 

The monograph on performance funding may become the most effective 
vehicle for disseminating results of the Kellogg Project to the higher edu- 
cation community beyond the campus of the University of Tennessee, Knoxville. 
In the meantime, the Project Director has made presentations to several out- 
side groups, including: 

a) The Knoxville Medical Auxiliary on November 4, 1983; 

b) The Southern Association of Colleges and Schools in New Orleans 
on December 13, 1983; 

c) The South Knoxville Kiwanis Club on March 12, 1984; 

d) The American Association for Higher Education in Chicago on March 
16, 1984; and 

e) The American Educational Research Association in New Orleans on 
April 24, 1984. 

Project Continuation 

The principal thrusts of the Kellogg Project — measuring student opinion 
and achievement in general education and the major field, and using the re- 
sulting information in decision-making and program improvement — will be con- 
tinued for the foreseeable future under the leadership of personnel associated 
with the Learning Research Center. Continuing these activities serves the 
interests of the University in two very important ways: 
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a) providing enhanced ability to gauge program quality for purposes 
of strategic decision-making and improvement, and 

b) bringing the University at least $3 million annually (or 5 percent 
of the E & G budget) in the form of a budget supplement from the 
State. 

The additional direct costs (i»e., above and beyond the expenses 
associated on an on-going basis with program accreditation, peer review, 
ability testing of freshmen, and departmental evaluative activities other 
than those specified by the THEC) of providing the outcome information speci- 
fied by THEC for an institution with the number of students and diversity of 
programs found at UTK is estimated at a little more than $100,000 per year. 

The Kellogg Project Director and the Director of the LRC will continue 
to enlist the counsel and assistance of the advisory conmiittaes, the Techni- 
cal Advisory Council, and the Instructional Evaluation Advisory Committee as 
needed in carrying out the data collection and related research programs and 
in communicating results to faculty. The members of the Technical Advisory 
Council will be particularly helpful in future efforts to put outcome infor- 
mation to use because most are associate deans charged with the responsibility 
of improving curriculum and instruction within their colleges. This responsi- 
bility gives the TAC members a vested interested in obtaining and using cur- 
rent outcome information. 

Project Impact 
Changes in Curriculum or Instruction 

The first change in academic requirements brought about by the increased 
campus interest in student outcomes was the adoption by the Faculty Senate in 
January 1983 of a statement for the University catalog concerning mandatory 
participation in "one or more evaluative procedures*' prior to graduation (see 
Appendix 7). This requirement was instituted to ensure that appropriate 
samples of students would take the ACT COMP exam in general education <^nd com- 
prehensive tests in the various academic disciplines. 

The score reports prepared by ACT following the testing of freshmen and 
seniors in 1983 with the COMP exam (see Appendix 11) have generated substan- 
tial interest among the academic deans, and among the department heads and 
faculty who have seen the reports. (More needs to be done to ensure that all 
faculty will be aware of the findings.) The measure of value added ~ score 
gain from freshman to senior years — provided encouragen^ent for curriculum plan- 
ners: Mean score gain at UTK is well above the mean of score gains at peer 
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institutions^ However^ percentile rankings on the two subscales Functioning 
in Social InstitULions and Solving Problems are not as high as the faculty 
would like to see them* \^ile there is no sound basis, nor is there an 
immediate need, for n.aking precipitous changes in general education distri- 
bution requirements on the basis of the COMP exam scores, there is consider- 
able interest in (1) analyzing relationships between COMP scores and such 
factors as courses taken, time on task, and participation in inteLnshlps, 
interdisciplinary courses, and other special experiences; and (2) reviewing 
future reports from ACT to see if the pattern of subscore strengths and weak- 
nesses obtained in 1983 is maintained in subsequent years. A group of faculty 
with interest in analyzing large data sets has been organized to carry out the 
analytical studies. 

Preliminary data analysers indicate that participation in student pro- 
fessional organizations \r> associated both with high acores and high score 
gain on the COMP exam. Tliis finding suggests, obviously^ that increased em- 
phasis on student participation in professional organizations might enhance 
the educational experience of students on campus. The UTK Coordinating Com- 
mittee on General Education, as well as the Board of Deans and others inter- 
ested in progran improvecnenl , ^ill continue to consider the COMP results and 
the unfolding picture of correlates of achievement as they contemplate broad 
changes in curricula and course work necessitated by increasing selectivity 
in admissions standards and a planned conversion from an academic calendar 
based on quarters to a semester system. 

Study of the COMP exam itself has proven to be a faculty development 
activity that holds much promise. Dr. Aubrey Forrest of ACT has been on 
campus twice under the auspices of the NCHEMS /Kellogg Project, and his presen- 
tations have been widely acclaimed by participants. The design of the COMP 
exam is quite interesting: Each item has been constructed to assess skills 
in a process area (communicating, solving problems, clarifying values) as 
well as a content area (social science, science/technology, the arts). More- 
over, the test items require the student to apply higher order intellectual 
skills such as analysis, synthesis, and evaluation, not just recognition and 
recall. Thus, careful study of the exam and the rationale on which it is 
based suggests to faculty ways of teaching and testing students that foster 
development of the more complex skills. Additional faculty development pro- 
grams of this kind are contemplated. 
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While changes in general education vill be made slowly due to the 
institution-wide implications of many such endeavors, changes in curricula, 
instruction, and supporting services associated with individual programs can 
be made more quickly and easily. The Van Liere-Lyons survey of enrolled stu- 
dents administered in Spring 1983 and Winter 198A (see Appendix 8), and the 
pilot projects implemented with Kellogg funds (see Table 1, page lA) already 
have resulted in more substantive changes than can be described fully in a 
brief presentation, and more changes are taking place with each passing week. 

Administration of the "Student Satisfaction Survey" by Van Liere and 
Lyons in Spring 1983 revealed some student assessments of the quality of 
programs and services that proved disquieting to administrators and faculty 
in several units. Students in the College of Communications provided rat- 
ings of "availability of advisor" and "availability of required courses in 
the major" that were below University means for those items. The Communica- 
tions dean was sufficiently concerned about these findings to take immediate 
steps to improve advising and make adjustments to give more students access 
to required courses. A full-time advisor for freshmen and sophomores was 
added to the dean's staff, and the average number of advisees assigned to 
each faculty member was reduced. 

Student Satisfaction Survey results in the colleges of business and 
Engineering, and the Department of Political Science prompted faculty in 
those units to initiate their own follow-up student surveys. The very de- 
tailed Engineering survey was administered by mail in February 1984, so 
results have not yet been compiled. In the business school a faculty member 
with experience in using a TELLUS machine that permits responses to ten 
items to be entered electronically and provides immediate tabulation of re- 
sults, administered questions to over 400 students during preregistratlon. 
That endeavor furnished evidence of some dissatisfaction with student advis- 
ing, and the appropriate associaue dean undertook a thorough review of ad- 
vising procedures and practices in the College. Advisors and all faculty 
have been sensitized to the needs for increased interaction with advisors 
and faculty that were indicated by students' responses to both the initial 
survey and the college follow-up. 

The department head and faculty in the Department of Political Science 
were concerned because the Spring 1983 administration of the "Student Satis- 
faction Survey" indicated that students were sorr;hat dissatisfied with the 



23 



quality of instruction provided by graduate teaching assistants. The depart- 
mental cor.jnittee on undergraduate instruction designed a follow-up study that 
involved administering to all undergraduate students taking political science 
classes in Fall 1983 the portion of the "Student Satisfaction Survey" dealing 
with the classroom experience. Following analysis of 1100 student responses 
the department head had, for -he first time, comparative data on the per- 
ceived teaching effectiveness of every faculty member and every graduate 
teaching assistant (GTA) • He shared the comparative data individually with 
each faculty member, and he believes this process will encourage faculty to 
take specific steps to improve their teaching. Indeed GTAs as a group did 
receive lower effectiveness ratings than faculty. Ways of responding to this 
finding are being explored — larger lecture sections taught by outstanding 
faculty, televised lectures, combinations of lecture by faculty with dis- 
cussion sections staffed by GTAs, Specific changes in content and methods 
of instruction are being planned for the introductory course in political 
science. 

The pilot project — a survey — carried out in the Department of Tex- 
tiles, Merchandising, and Design pointed to a need, once again, for improve- 
ment in student advising. In response a new curriculum planning sheet for 
majors was designed. 

In several units the results derived from pilot projects confirmed 
present practice, e.g., high scores on the Graduate Record Advanced Teats 
by seniors in the departments of psychology, history, botany, and micro- 
biology encouraged those faculties to continue current patterns of course 
work and classroom instruction. Rut in the departments of Geography and 
Food Technology and Science, the faculty decision to design their own compre- 
hensive exam for seniors had profound effects. In order to determine the 
content of their exams the faculties had to cor Ider in a more intensive way 
than heretofore, such curriculum matters as the relative emphases given to 
specialty areas of the discipline, the way course sequences fit together 
and build on each other, and the competencies students should possess upon 
completion of the curriculum for program majors. 

The exam developed over the better part of a year by the faculty of 
the Department of Geography has four sections: Physical Geography, Economic 
Geography, Cultural Geography, and Technique, Seniors who took the exam when 
it was administered for the first time in February 1984 attained high scores 
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on the Cultural Geography section; their lowest scores were in economic 

geography* These specific findings and che global picture of scores have 

caused the faculty to undertake several curriculum changes: A course in 

economic geography will be added, cultural geography will be refocused, and 

students will be encouraged, perhaps required, to take a stronger common core 

of courses. In summarizing his perception of the effects on the department 

of designing the exam and using the findings in program evaluation, Sidney R* 

Jumper, head of the Department of Geography, wrote: 

The greatest benefit of the project has been that 
faculty members have been forced to consider the 
curriculum from the standpoint of measurement of 
speci ic outcomes — or the quality of the student 
product — rather than just in terms of general 
objectives that often reflect compromises of di- 
vergent views. In my view the benefits are vastly 
in excess of the cost of the project (in geography) 
in time, faculty resources, and money. 

As in Geography, all faculty in the Department of Food Technology and 
Science (a unit in the College of Agriculture) were involved in developing 
theii comprehensive exam for seniors. \^ile Geography faculty chose not to 
involve in the design phase UTK faculty with expertise in measurement, the 
Food Tech faculty did schedule several sessions with a Kellogg Project con- 
sultant identified for this purpose. Professor Schuyler Huck from the De- 
partment of Educational Psychology and Counseling. Dr. Huck provided gen- 
eral guidance in developing multiple choice test items, and assisted in 
determining instrument reliability. In addition to the opportunity for 
internal review of instruments by a specialist in measurement, both ^he de- 
partments were encouraged to retain two external consultants to review their 
tests. The two faculties identified respected scholars in the discipline and 
asked those individuals to review drafts of their exams and even to give the 
tests to their own students for purposes of obtaining comparative data. 

The test designed by faculty in Food Technology and Science is comprised 
of six sections: Microbiology, Food Chemistry, General, Meats, Dairy Products, 
and Crop Products. It was first administered to seniors in June 1983, five 
months after the faculty began work on it. In one respect the first admin- 
istration of the exam was an effort to collect base-line information for 
evaluation of the effects of a new course distribution requirement. Pre- 
viously students had had the option to select whatever combinations of courses 
they wanted from offerings in the three areas: Meats, Dairy Products, and 



Crop Products. In the summer of 1983 a curriculum change took effect that 
required each departmental major at the undergraduate level to take at least 
one course in each of the three areas. Subsequent administrations of the 
exam will provide evidence of the effectiveness of this requirement in broad- 
ening the knowledge of majors concerning the field of food technology. 

Analysis of students' test scores also brought about some iraniediate 
changes in the Food Tech Department. Faculty were not satisfied with the 
scores achieved on the Microbiology and Food Chemistry sections of the test. 
Students were not able to apply their knowledge to solve problems to the ex- 
tent faculty had hoped they would be able to do. Following a series of meet- 
ings, the faculty teaching microbiology and food chemistry courses agreed to 
place much more emphasis on applications in their teaching and in their class- 
room tests. 

The Food Tech faculty already has begun to consider ways of improving 
their exam for seniors. They want to add a performance measure that would 
take students into the laboratory to solve certain problems. Students' lab 
techniques would be judged as well as their approach to a problem and their 
resolution of it. 

In summarizing his perception of the importance of the test design 
project, the head of the Department of Food Technology and Science, J. T. 
Miles, wrote: 

We plan to share our progress with the Education 
Committee of our international professional or- 
ganization and try to develop (the exam) further. 

Faculty believe the time and resources necessary 
for developing the exam were worthwhile invest- 
ments. Among other things, this experience has 
increased our competence as developers of class- 
room tests. 

The use of nationally standardized exams to test student achievement 
for purposes of program evaluation can be valuable. At DTK the dean of the 
College of Business asked seniors to take the Business Assessment Test 
offered by Educational Testing Service as part of its Undergraduate Assess- 
ment Program. Students' scores gave evidence of particular strength in 
economics and weakness in business law. The college faculty had other 
Indications of these anomalies in the curriculum and acted to reduce the 
three-course requirement in economics to two courses so that a requirement 
In business law could be added. Notwithstanding this kind of example, the 
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experience -^n this campus to date indicates that faculty have a greater in- 
tellectual stake in the outcomes of testing, and are more likely to under- 
take improvement initiatives based on those outcomes, if they have invested 
the time to become involved in designing their own comprehensive exam. 

On technical grounds one can argue against the use of locally developed 
tests; there are no norms against which to judge local performance, reliabil- 
ities may be questionable, content and predictive validity are difficult to 
demonstrate. But when a test is being used to assess and improve program 
quality , students' scores are aggregated and faculty consider the implica- 
tions of scores for changing the program rather than for making judgments 
about the relative competence of individual students . Thus the importance 
of technical flaws in the instrument is minimized, and the potentiality for 
effecting meaningful program improvement is great. 
Changes in Student Services 

Most of the substantive changes in student services made at UTK as a 
result of the increased emphasis on using student outcome information in pro- 
gram evaluation have taken place in academic services at the college or de- 
partmental level. Analyses of the data from the Student Satisfaction Survey 
indicated the great importance to students of good advising apd increased 
interaction with faculty. Student participation in professional organiza- 
tions was found to be a significant correlate of achievement in general edu- 
cation as measured by the COMP exam. These findings were put together in 
several colleges to provide impetus for improving advising and increasing 
interaction between students and faculty both in the instructional context 
and in strengthened student professional organizations. Descriptions of 
some of these actions in the several colleges were given in the preceding 
section. 

The University requirement that students participate in "one or more 
evaluative procedures*' prior to graduation ensures that some students will 
participate in the assessment of achievement in general education via the 
COMP exam or achievement in the major field via nationally standardized or 
locally developed tests. Those seniors who are not asked to take an exam 
in their senior year (testing for all seniors is too expensive, and unneces- 
sary) are required to complete the Student Satisfaction Survey. An area of 
the Career Planning and Placement Center has been identified as the site for 
seniors to complete the Survey form. Some seniors who would not have availed 
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themselves of the services of the placement center have been brought to the 
center by the evaluation requirement and have stayed to file an employment 
resume'T 

Responses to the Student Satisfaction Survey items concerning regis- 
tration and availability of courses indicated some dissatisfaction with the 
registration process. The Vice Chancellor for Student Affairs, Howard 
Aldmon, used this finding to support a decision to establish an earlier cut- 
off date for admission to the University so that student demand for classes 
could be assessed in a more timely fashion and extra sections scheduled to 
accommodate demand. Ir a further attempt to improve the quality of student 
advising, Vice Chancellor Aldmon asked that each dean provide a representa- 
tive to meet with students encountering special problems during the drop/add 
process. 

Student retention is a matter of increasing concern to campus admin- 
istrators across the country. Analysis of Student Satisfaction Survey 
responses for students returning to the University and those not returning 
two quarters later (see Appendix 10) has provided preliminary evidence that 
satisfaction with the University is an important factor in students' deci- 
sions to pursue their studies at UTK, Higher grade point averages and fewer 
hours of employment (less than 30 hours per week) also are linked with per- 
sistence on this campus. Developers of the Student Satisfaction Survey, 
Kent Van Liere and William Lyons, found that patterns of student satisfac- 
tion varied by college. As a consequence, they have provided a profile for 
each of the nine colleges enrolling undergraduates of factors characterizing 
student academic and social satisfaction in that unit. For some colleges 
social satisfaction — interaction with peers (perhaps in student profes- 
sional organizations) or membership in a social fraternity — contribute 
most to overall satisfaction with the University, For the professional 
schools — nursing, business, architecture, engineering — interaction with 
faculty is a potent factor in determining satisfaction. These profiles are 
being used by college faculties to make adjustments they hope will help to 
increase student retention. 

Changes in Institutional Planning and Evaluation Activities 

In June 1982 the NCHEMS/Kellogj^ Project Technical Advisory Council 
recommended that steps be taken to incorporate student outcome information 
in the on-going academic program review process to ensure that every academic 
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unit would have an impetus to consider such information in program evaluation 
at least once every five years^ The Council further recommended that infor- 
mation from program reviews be considered in budgeting and strategic planning. 

The University of Tennessee, Knoxville has a well-established (begun in 
1974), carefully crafted program review process that is conscientiously imple- 
mented by the Provost and his staff. Central administrators, including the 
Chancellor himself, review the self-study, and reports prepared by internal 
and external reviewers following a 2?$-day intensive site visit. Objectives 
and aspirations of the department or program under review and the recommenda- 
tions of the reviewers are considered in University-wide as well as unit plan- 
ning and in internal resource allocations. 

Prior to 1983 the UTK program review guidelines, like those at many other 
institutions, focused primarily on program input factors as evaluative cri- 
teria. Qualifications of faculty, quality of students as measured by aptitude 
tests or grade-point averages, the funding base, and adequacy of facilities 
and the library collection, were included by program faculty in the self-study 
and examined by reviewers. In July 1983 the UTK Kellogg Project Director was 
given the opportunity to submit new material for the program review guide- 
lines that would provide a focus on student outcomes such as placement of 
graduates, opinion concerning quality of the program and supporting services, 
and achievement in general education and the major field. The revised guide- 
lines, with new material indicated, appear in Appendix 2; they were put into 
effect in September 1983. 

To emphasize the Importance of using student outcome information in pro- 
gram evaluation, in January 1984 Provost George Wheeler asked every academic 
unit to develop a plan for assessing student achievement in the major field 
at least once in the next five years. 

In an additional development, the Kellogg Project Director began serv- 
ing in Spring 1983 as a member of the central administrative team that con- 
ducts the program reviews and related follow-up activities. This give3 her 
an opportunity to call attention to the evidence provided by outcome informa- 
tion throughout the review process. 

The UTK Planning and Budgeting Coordinating Coiranittee, co-chaired by 
Provost Wheeler and Executive Vice-Chancellor for Business, Planning, and 
Finance Homer Fisher, is less than two years old; Chancellor Jack Reese 
appointed the group in September 1982. This committee has reviewed the Van 
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Liere-Lyons retention study (Appendix 10) in preparation for planning strate- 
gies to increase student persistence at UTK* The information on freshman and 
senior COMP exam scores (Appendix 11), accompanied by preliminary analyses of 
correlates of achievement in general education ~ including reasons given by 
freshmen for choosing to attend UTK has been considered by the committee 
as it seeks ways to clarify the mission statement of the University and com- 
municate it to the clientele the institution hopes to serve. During Spring 
Quarter 198A the committee will work with Professors Van Liere and Lyons and 
the Learning Research Center staff to develop and administer a survey of 
alumni to obtain their opinions concerning program quality. The committee 
has begun to use the academic program review documents, with their new 
emphasis on student outcome information, in determining which programs to 
strengthen through allocation of additional resources, and which to combine 
with others or terminate. 

In the most tangible and immediately effective illustrations of out- 
comes utilization to date. Executive Vice Chancellor Fisher inserted a 
reference to the use of student outcome information as evidence of program 
quality in the 1984 instructions for use by program heads in preparing their 
annual budget requests (see Appendix 12). In addition, the Planning and 
Budgeting Coordinating Committee used student outcomes as one of the criteria 
in selecting the campus proposals to be entered in Tennessee Governor Lamar 
Alexander's 1984 state-wide Centers of Excellence competition. Previously 
the criteria for assessing quality in procedures such as these had included 
only input and process variables. 

Conclusion 

The use of student outcome information in program assessment and improve- 
ment has been woven into the fabric of institutional planning and decision- 
making during the two years of the NCHEMS/Kellogg Project at the University 
of Tennessee, Knoxville. As a result of the adoption of appropriate new 
policies and procedures, program heads in all colleges are using outcome data 
to inform the process of improving their programs, while central administra- 
tors are considering these data as they make strategic decisions about pro- 
gram mix and allocation of internal resources. 

Many institutions employ systematic methods to collect information on 
student achievement and opinion. The UTK program is unique in its scope: 
It attempts to provide mutually supportive combinations of outcome information 
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and to explicitly encourage use of such information in improving programs in 
each of the 111 disciplines represented at this research university of 26,500 
students* 

Until recently the strong tradition of the academy to manage its own 
affairs deterred public vvoWng about the efficiency and effectiveness of 
programs and services in higher education. Now the loss of confidence in the 
American educational system, coupled with the need to make better infomed 
decisions about the allocation of scarce financial resources among a variety 
of important services, has stripped away any imip.unity from public scrutiny 
that colleges and universities may have enjoyed in the past* The ability to 
conduct comprehensive program evaluations — comprehensive in that programs 
are monitored from the setting of objectives, through the allocation and 
utilization of resources in implementation, to the measurement of outcomes 
related to the objectives — is becoming an institutional imperative in 
higher education. 

State legislatures from California to Missouri to Florida are expres- 
sing interest in alternatives to enrollment-based funding of higher educa- 
tion that reward institutions for providing programs of exceptional quality. 
The Southern Association of Colleges and Schools is considering the use of 
outcomes assessment in its accreditation standards. Tennessee has become 
the first state to utilize performance criteria in allocating a portion of 
state funds for higher education. 

The leadership of Chancellor Jack Reese, Executive Vice Chancellor 
Homer Fisher, and Provost George Wheeler and his staff; the information needs 
of the Planning and Budgeting Committee; a healthy attitude toward innovation 
among department heads and faculty; the impetus of the NCHEMS /Kellogg Project; 
and the financial incentive provided by the State's performance funding approach 
have proven to be the key factors in the successful effort to promote the use 
of student outcome information in program evaluation and improvement and in- 
stitutional planning and decision-making at the University of Tennessee, 
Knoxville. 

The task of convincing administrators and faculty of the value of syste- 
matically collecting evidence of program quality and putting it to use re- 
quires strong leadership, and time. The importance of the NCHEMS/Kellogg 
Project in providing access to these resources at UTK must be emphasized. 



Powerful objections to the collection and use of outcome information can 
be raised: The process costs too much, faculty have difficulty agreeing on 
program objectives, appropriate instruments are not available, and instru- 
ments that are available lack acceptable reliability. Representative study 
and planning groups must be established to consider these objections, to re- 
view measurement methodologies, and to assess the potential benefits of employ- 
ing these strategies. The methodologies must be adapted to fit institutional 
and program missions. Individuals participating in the study groups must 
communicate their findings to the students, faculty, and administrators they 
represent. Ultimately all must come to the realization thct if faculty be- 
lieve an instrument can give them information about strenghs and weaknesses 
of their program, some of the technical imperfections in the instrument may 
be overlooked since data will be aggregated for assessment of the program 
rather than the relative competencies of individual students . Conducting 
pilot tests of alternative methodologies will provide an experience base 
upon which widespread application can be built. 

At UTK the NCHEMS/Kellogg grant provided the leadership, the time, and 
some financial assistance for (1) carrying out the studies essential to 
establishing the institution's ownership of its student outcomes assessment 
program, (2) establishing the communication networks necessary to inform 
faculty and students of the benefits of such a program, and (3) pilot test- 
ing alternative methodologies for collecting and using outcome information* 
Thus the institution has come to terms with the requirements of the State's 
instructional evaluation program, has adapted that program to serve its own 
needs, and new is in position to take full advantage of Tennessee's pio- 
neering effort to allocate state funds on the br^sis of performance criteria. 
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INSTRUCTIONAL EVALUATION VARIABLE 
PROGRAM ACCREDITATION 

Purpose 

This variable is designed so as to reward institutions that design and offer 
academic programs, for which accreditation services are provided, that meet 
or exceed the standard of responsible accreditation agencies. 

Performance Standard and Point Allocation 

An institution may be awarded up to 25 points on this variable. The number 
of points awarded to the institution will be a percentage of this maximum 
amount calculated as the percentage of eligible programs accredited. 

Definitions and Procedures 

(1) A "program" is defined as a sequence of educational experiences leading ** 
to a degree major as listed in the THEC program inventory. 

(2) A program is defined as "eligible" if there is a COPA member agency or 
organization which accredits programs for that field and degree level 
(unless exempted under (6) below). Additional accrediting agencies may 
be proposed by governing boards ♦ Upon THEC staff approval, all programs 
accreditable by such agencies will be included as eligible statewide, 

(3) Program fields covered by an umbrella accreditation will not be counted 
as "one" unit, but each degree major as "one." For example, if an 
institution offers five bachelor's degree majors in business, and the 
business school or college is AACSB accredited at the undergraduate level, 
these five programs will be counted as five programs for the purposes of 
this variable. 

(A) Programs automatically excluded from the list of eligible programs are 

programs (a) that have been approved by the THEC for less than five years, 
unless the program is Accredited by a COPA agency, (b) that are being 
terminated or phased out — based on appropriate official action, and 
(c) that have been identified as "inactive" by the appropriate board and 
the THEC. 

(5) A program eligible for accreditation by more than one agency will be 
counted only once on the eligible list. 

(6) Where program accreditation efforts are shown to be unjustified on a 
statewide basis in relation to an accumulation of factors such as economic 
feasibility, critical mass of enrollees, low benefits to students, more 
important qualitative priorities, etc., institutions may request 
res.pective governing board to seek program exception. Any exception 
approved by the THEC staff must apply to all similar program areas in the 
state «, 
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Proposals from governing boards for statewide changes in eligibility 
of programs or appropriateness of accrediting agencies as outlined in 
(2) and (6) above must be 'submitted to the THEC staff before January 1 
each year to facilitate any necessary revision of the eligible program 
or acceptable accrediting agency lists for the next budget cycla* The 
official list of eligible programs or appropriate agencies shall be 
maintained by the THEC staff based on inventory records and approved 
exceptions as noted above* 
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II 

INSTRUCTIONAL EVALUATION VARIABLE 
PROGRAM FIELD EVALUATION 

Purpose 

This variable consists of two standards • An institution nay earn a maximum 
of 10 points under the first standard (IIA) and a maximum of 20 additional 
points under the second standard (IIB), The first standard is intended :o 
encourage an institution to evaluate the quality of each of its academic 
programs at least once within a five year period. The second standard is 
designed to reward those institutions whicn can demonstrate on the basis of 
test results that the quality of their programs is increasing or has attained 
an above average level of quality. Together, these standards provide a means 
of evaluating the quality of the specialized academic offerings of institutions. 

PROGRAM FIELD EVALUATION 

IIA 

EXTERMALLY VALIDATED TESTS, LOCALLY DEVELOPED TEST, OR EXTERiTAL i'EER REVIEW 
Performance Standard and Point Allocation 

Under Standard IIA, an institution tray be awarded up to 10 points. The number 
of points awarded to the institutions will be a percentage of this maximum 
amount calculated as the percentage of program fields which have met the 
requirements outlined below within the past five academic years.* 

The institution has assessed the performance of a representative sampling of 
graduates of the program field by means of an externally validated instrument 
approved by the THEC staff. This instrument shall be applied to and appropriate 
for the program level which has produced the largest number of graduates in recent 
years at that institution. 

OR 

The institution has assessed the performance of a representative sampling of 
program field graduates by process of the administration cf a locally developed 
program test. This instrument shall be applied to and appropriate for the 
program level which hps produced thp largest number of graduates in recent years 
at that institution. 

OR 



40 



RJC 



Tlie institution has evaluated the quality of individual programs via 
external peer review (this alternative is not available for sccraditsbls 
programs). This evaluation shall cover all levels of the program field 
offered by the institution. 

*For the first four years' administration of this variable, 
the 10 points maximum will be awarded to an institution 
according to the following schedule. 

A. In the first year-- at least 207. of the program fields 
have met one of the listed requirements within the first 
year, 

B. In the second year—at least 40Z of the program fields 
have met one of the listed requirements within the first 
or second year, 

C. In the third year—at least 60.1 of the program fields 
have met one of the listed requirements within the first, 
second, or third year. 

D. In the fourth year— at least 80Z of the program fields have 
met one of the listed requirements within the first, second, 
third, or fourth year. 

IIB PROGRAM FIELD EVALUATION 



IMPROVED PROGRAMS OR PROGRAMS OF EXCEPTIONAL QUALITY 



Perforniance Standard and Point Allocation 



To be eligible for points under this standard, an institution must demonstrate 
that it has assessed the performance of a representative sample of graduates 
of its program fields via externally validated tests or locally developed 
tests. Up to 20 points may be awarded under Standard IIB. The number of 
points awarded to the institution will be a percentage of this maximum calcu- 
lated on the basis of the percentage of programs that have met the requirements 
outlined below within the past five academic years, (See the table below for 
further details,) 

The institution can demonstrate on the basis of an externally validated test, 
appropriate fco that field that the performance of program graduates exceeds 
the norm* 

OR 

The institution can demonstrate on Che basis of an externally validated test 
appropriate to that field that the performance level of program graduates 
exceeded the level of performance by program graduates on the most recent 
administration of that test. 

OR 



The institution has assessed the performance of a representative sampling of 
program graduates through administration of a locally developed test and can 
demonstrate program graduate scores which exceed the scores from the most 
recent previous administration of that test,** 
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AWARDS UNDER STANDARD IIB 



Percentage of Program Fields Points 
Meeting Requirements - Awarded 



75% 


- 


100% 


20 


72.5% 


- 


74.9% 


19 


70.0% 


- 


72.4% 


18 


67.5% 


- 


69.9% 


17 


65.0% 


- 


67.4% 


16 


62.5% 


- 


64.9% 


15 


60.0% 


- 


62.4% 


14 


57.5% 


- 


59.9% 


13 


55.0% 


- 


57.4% 


12 


52.5% 




54.9% 


11 


50% 




52.4% 


10 


47.5% 
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9 


45.0% 




47.4% 


8 


42.5% 




44.9% 


7 


40.0% 




42.4% 


6 


37.5% 




39.9% 


5 


35.0% 




37.4% 


4 


32.5% 




34.9% 


3 


30.0% 




32.4% 


1 


27.5% 




29.9% 


1 


0 




27.5% 


0 



**In order to compensate Institutions for the initial costs of developing 
local tests, institutions will be rewarded for the first administration of 
such tests during the first five years (to July 1, 1988). A locally 
developed test administration for the first time in this period will be 
scored as if the institution's test scores had exceeded a previous score on 
the same test. 

Definitions and Procedures 

(1) In general, a "program field" is defined as all levels of programming 
bearing the same name as an academic major. A group of closely related 
programs with dissimilar names may also be considered a single program 
field. General technology and general transfer programs leading to an 
associate degree are exempt from this variable as are pre-prof essional 
programs which do not result in a degree under that name. All individ- 
ualized programs offered by an institution will count as one program 
field for purposes of this variable. Programs automatically excluded 
from consideration under this variable are programs (a) that have been 
approved by the THEC for less than five years, (b) that are being 
terminated or phased out — based on appropriate official action, 
(c) that have been identified as "inactive" by the appropriate board 
and the THEC, and (d) that are offered at a level below the bacca- 
laureate at baccalaureate degree granting institutions (nursing pro- 
grams excepted). Program fields which are accredited under Variable I 
and which are performance oriented shall not be included in this 
variable. A list of progra:a fields for each institution shall be 
maintained by the THEC staff. 
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An institution choosing to conduct external peer reviews must submit a 
plan for external review through its governing board staff to the THEC. 
staff for approval prior to the review. The plan for external peer 
review must include names and vita of at le^st two peers proposed to 
conduct the review, a schedule of planned activities to be included in 
the review, include efforts to measure the improvement of educational 
outcomes to the maximum extent possible, and provide for a written 
critical report summarizing the findings which will be forwarded to the 
THEC as part of the budget request process. 

A "locally developed program test" must be constructed in cooperation 
with at least one similar institution with a similar degree major 
program or in consultation with a team of at least two external consul- 
tants, at least one of which must be an expert in the content of that 
program field. 

An institution choosing to use locally developed program tests must submit 
a plan for test construction through its governing boads staff for THEC 
staff approval prior to construction. The plan for test construction 
must include a schedule of activities, sampling procedure, credentials of 
cooperating institution staff or credentials of external consultants, and 
a proposed schedule for submission for THEC staff approval prior to use. 
Results and analysis of locally developed program tests must be submitted 
as part of the budget request process. 

The master list of appropriate externally validated tests available for 
programs will be determined and maintained by the THEC staff. 

In choosing among externally validated tests, locally developed tests or 
peer reviews, an institution should consult its governing board staff. 

In reporting test results under Standard IIB utilizing program field tests 
for which more than one datura descriptive of average performance for both 
the sample of graduates and the norm group is available, a mean score will 
be accepted in preference to a percentile datum and a percentile datura will 
be accepted in preference to a pass/fail rate. If more than one norm 
group is available for comparison, national norms are preferred to regional 
norms which are preferred to state norms. 

The awarding of points under both IIA and IIB is on the basis of "official 
test" scores. Once a test is given and points awarded under IIA, the score 
reported becomes the "official test" score for five years or until an 
institution notifies the THEC staff that it intends to "retest" that field 
within the five-year period. Intent to retest must be declared in advance 
of such testing and the results must be reported to the THEC. The score 
of the retest becomes the "official test" score and is utilized for all 
point calculations under variable II. 

Reference to the "most recent previous administration" of a test refers to 
the most recent administration of an "official test" (defined above). An 
exception to this is made for the first test of a program field qualifying 
as .an "official test". 
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INSTRUCTIONAL EVALUATION VARIABLE 
INSTITUTION-WIDE EDUCATION OUTCOMES 



Purpose 

This variable consists of two alternative standards. The particular 
standard to be applied is dependent on the class of institution. This 
variable provides a means of evaluating the genersil (non-program-specif ic) 
quality of the educational program at each institution. 

IIIA GENERAL EDUCATION OUTCOMES 

This standard shall apply to all four year institutions and may apply to 
community coll'eges as described below. 

Performance Standards and Point Allocation 



(1) The institution will be awarded 5 points if, within the past five academic 
years, the institution has assessed the performance of a representative 
sample of graduates for its major academic degree utilizing the ACT-COMP 
Objective or Composite measure, 

(2) The institution will be awarded an additional 20 points if through 
annual assessment utilizing the ACT-COMP measure, the institution can 
demonstrate that the performance of its graduates regarding value added 
is above average when compared with the performance of graduates of 
comparable institutions, (See definition //6 for procedure.) 

OR 

The institution will be awarded an additional 20 points if through annual 

assessment utilizing the ACT-COMP measure, the institution can demonstrate 
an improvement in value-added from the most recent institutional measure 
of value-added, (see definition if/6 for procedure,) 

IIIB PLACEMENT OF GRADUATES 

This standard shall apply to all technical institutes. Community colleges 
must first make a determination as to which of their programs can be 
assessed by a measure of general education outcomes. For these programs, 
the standards of IIIA shall apply. For the remaining programs at these 
institutions, TUB shall apply. The total number of points awarded shall 
be prorated between the two standards according to this division. 

Performance Standards and Point Allocation 

(1) Fo,r programs being evaluated within this standard, the institution will be 
awarded 5 points if the institution each year has conducted a follow-up 
survey of all graduates to ascertain their emplo>'ment status in the cluster 
of occupations for which they were trained. 
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(2) The institution will be awarded an additional 20 points if, through 
analysis of the surveys conducted in IIIB(l), the employment rate for 
graduates in the cluster of occupations for which they were trained 
exceeds 70 percent. 

OR 

The institution will be awarded an additional 20 points if the employment 
rate of graduates in fields for which they were trained exceeds the 
employment rate in the most recent similar survey of employment rate of 
graduates in fields for which they are trained. 

Definitions arid Procedures 

(1) Follow-up surveys must be designed to establish the employment status 
of all program completers during a period not earlier than 30 days 
following program completion and not later than 90 days following 
program completion. The single exception to this shall be the survey 
of students completing in the spring quarter in time for a May or June 
convocation. These students must be surveyed not earlier than September 1 
and not later than October 31 following their program completion. 

(2) All completers surveyed within a fiscal year will form the basis of 
calculation of employment rate. The placement percentage is calculated 
as the ratio of the total number of students placed in fields for which 
they were trained to the total number of program completers less those 
in military service or pursuing further education. 

(3) A list of '^clusters of occupations" appropriate to each program subject 
to evaluation under i^tandard IIIB shall be maintained by the THEC staff. 

(4) A representative sample is a sample of entering students or graduates 
chosen so that the sample statistically represents the population of 
entering students or graduates for a given year. 

(5) Value added shall be measured by a comparison of the general education 
mean score as measured by the ACT or COMP for entering freshmen to the 
mean COMP score for a graduating class. Any one of the following 
procedures may be used: 

(a) Longitudinal Study using the COMP Composite Examination 

(b) Longitudinal Study using the COMP Objective Test 

(c) Cross-sectional Study using the COMP Composite Examination 

(d) Cross-sectional Study using the COMP Objective Test 

(e) Exit-level assessment only, estimating the entry level 
COMP score based on a concordance table with the ACT 
composite score. 

(6) Above average perfoniiance in value-added must be demonstrated by an 
institution having a value-added mean score which exceeds the value-added 
mean score for a similar set of institutions measuring value-added, with a 
comparable procudure. A similar set of institutions shall nimiber no less 
than six and shall include, to the extent possible, institutions with 
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similar purposes, similar enrollments, similar support systems, 
and similar testing or surveying techniques. A similar set of insti- 
tutions cannot be exclusively or predominantly composed of in-state 
public institutiotic. The THEC staff shall determine which set of 
institutions are to be considered similar following consultation with 
institutional and governing board staffs and with personnel from the 
American College Testing Program. 

The sampling procedure for activities in this variable must be submitted 
prior to use for THEC staff review. 

Institutions must submit a written report including scores, survey 
results and analyses as part of the budget request process. 

Calculations of value added shall be rounded to the nearest whole number 
and point allocations made on that basis. 
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IV 

INSTRUCTIONAL EVALUATION VARIABLE 
INSTRUCTIONAL IMPROVEMENT BASED ON REFERENT GROUP SURVEY 

Purpose 

This variable consists of two standards, A maxitnum of 5 points can be earned 
under each standard for a total of 10 points under this variable. This 
variable is designed to encourage institutions to seek evaluation of its 
overall academic program quality by consumers of the educational product. 

IVA SINGLE SURVEY 

Performance StanOc^^ds and Point Allocation 

The institution will be awarded 5 points upon demonstration that the institution 
has surveyed, with an evaluative instrument, a representative sample of at least 
one of three referent groups (enrolled students, forrnerly enrolled students, 
or community members/employers) with application to the majority of its program 
fields or to the entire institution. To be awarded points for this standard, 
the institution must submit a brief presentation of the analysis of the survey 
results and provide a description of specific substantial, instructional improve- 
ment actions taken as a result of the survey and analysis when such improvement 
actions are indicated. 

IVB TWO SURVEYS 

The institution will be awarded an additional 5 points if the institution has 
surveyed, with an evaluative instrument, two or more of the referent srouos with 
application to the majority of its program fields or to the entire institution* 
To be awarded points for this standard, the institution must submit a brief 
presentation of the evaluative survey results and provide a description of 
specific, substantial, instructional improvement actions taken as a result of 
the surveys and analyses when such improvement actions are indicated. 

OR 

The institution will be awarded an additional 5 points if the institution can 
demonstrate that an evaluative survey has been administered more than once to 
the same referent group and can demonstrate for this referent group's most recent 
evaluation, the institution has received improved scores from the previous 
survey taken as a whole. 

Definitions and Procedures 

(1) An ''evaluative survey'* is defined as one yielding quantifiable indices 
reflecting satisfaction with or evaluation of instructional programs. 
The survey instrument may be a nationally or locally constructed instru- 
ment* A list of acceptable instruments for this variable will be main- 
tained by the THEC staff* Prior approval by the THEC staff for the use 
of instruments not on this list is required. 
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(2) 
(3) 



(A) 
(5) 

(6) 



A representative sample means a sample chosen so that it statistically 
represents the population. ^ 

Instructional improvement actions must relate directly to improvement 
of classroom instruction or indirectly in terms of academic support 
activities such as library services, academic counseling services, etc. 
Utems such as food service, parking, or other student conveniences, 
etc. , are excluded) . 

As part of the budget request, an institution must submit a copy of the 
survey instrument, date(s) of administration, description of sampling 
procedure, and analysis sufficient for any points claimed. 

^sp^^M^?^•^^ "'^ " necessary that a single instrument be 

used. Multiple instruments employed within the same fisc-1 year consti- 

Se in.H^^• ^ ^^^^ ^PP^^^ ^ "Majority of 

the institutional program fields or co the entire institution. 

To be awarded points under this variable, the survey or surveys must be 
conducted during the fiscal year immediately preceding the fall 
appropriations request cycle. 
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V 

INSTRUCTIONAL EVALUATION VARIABLE 
PLANNING FOR INSTRUCTIONAL PROGRAM IMPROVEhENT 

Purpose 

This variable consists of two standards. A maxiimim of 5 points can be 
awarded under each standard for a total of 10 points under this variable. 
This variable is designed Co encourage institutional self-evaluation of its 
academic program quality. 

VA PLAN DEVELOP>ENT 

Performance Standards and Point Allocation 



I 
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The institution will be awarded 5 points under this standard, provided it 
submits an acceptable annual plan for instructional improvement in the forth- 
coming year to the THEC staff at the time of submission of appropriations 
requests for that forthcoming year. An acceptable plan must exhibit these feature! 

a. Specific goals and benchmarks or measurable objectives can be | 
reached in the planning period are set forth. 

b. Activities scheduled as part of the plan must provide for an 
evaluation component. 

c. All activities which form the basis of claims for points under the 
other four institutional evaluation variables should be included in 
the plan, but the plan should address additional neans of instructional 
improvement . 

d. Faculty must be involved in the development, execution, and evaluation 
of the plan. 

e. The plan must be focused upon improvement in instruction, either directly 
in terms of improved classroom performance as illustrated by outcomes 
measures or indirectly in terms of improvement to academic support 
activities such as library services, academic counseling services, etc. 

f. The plan should be consistent with longer term plans of the institution, 
its governing board, and the TIEC. 

VB PLAN EVALUATION 

Performance Standards and Point Allocation 



The institution will be awarded an additional 5 points under this standard, 
provided it submits at the time of submission of appropriations requests an 
evaluation of the plan for instructional improvement covering the previous year. I 
This evaluation must report the degree to which the plan was executed and the 
results obtained in terms of reaching goals and benchmarks or measurable objective 
and completion of activities. Only those institutions which can demonstrate that j 
least half of the obiectives and benchmarks have been reached or activities ; 
favorably evaluated will be awarded points under this standard. 




APPENDIX 2 



Academic Program Review Guidelines 
for the 
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SELF-STUDY DOCUMENT 



A self-study document is prepared by the acadernic unit prior to the 
review. Copies are distributed to members of the review team at least two 
weeks prior to the on-canpus evaluation. This outline for the self-study 
document is designed to provide guidelines and assist the unit, not to 
prescribe a rigid format for its content. 



Functi on 

Specify clearly the primary function of the academic unit, including 
immediate and long-range goals and/or objectives for instruction , research, 
and public service, within the broader context of the college and the total 
uni versi ty • 



Program 

Provide a brief statement describing the aca'''^mic program, including: 
role, scope, breadth and depth. Also, describe the program and its 
components in terms of emphasis on preparation for teaching, research 
and/or professional practice. Assess the demand for the program in the 
community, state, and/or region. 



Faculty 

Provide a brief vita for each faculty member. Describe the faculty in 
terms of strengths and weaknesses, particularly as related to the role and 
scope of the academic program. Include information in the summary 
concerning the past three years for the following: 

1. Special teaching, research or professional practice awards to the 
faculty. 

2. Publications in referred journals. 

3. Books published. 

4. Monographs or manuals published. 

5. Journals edited or number of faculty members serving on editorial 
boards. 

6. Grants and contracts awarded from agencies external to the 
University. 

7. Presentations at national meetings. 

8. Number of faculty approved to teach 6000 (doctoral) level courses. 



*1983-84 Additions Un derlined 
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9. Number of faculty approvsd to direct doctoral dissertations. 

10. Any information about faculty quality collected from students or 
al umni , 

11. International experience. 

12. Faculty development activities. 



Students 

Provide student information for the past five years. Include any 
information considered appropriate, but aTTeast the following if 
available: 

1. Number of applicants. 

2. Number of students admitted. 

3. Number of women, minorities and international students enrolled. 

4. Number of students graduated. 

5. Number of students who dropped out. 

6. An assessment of the quality of students as indicated by test 
scores, grade-point averages, or other data. 

7. Financial assistance available, including number of students 
awarded assi stantships and fellowships. 

8. Description of student recruitment procedures. 



Library 

Provide an assessment of the adequacy of library holdings for the program. 



Physical Facilities 

Provide a brief summary of the physical facilities and describe their 
effect on the academic program. Include a statement concerning any 
pertinent equipment needs. 



Program 

1. Admission procedures— Describe how students are selected for the 
program. 

*2. Statement of desired outcomes of instruction for students. 

3. Innovative, unique, or outstanding features of the program. 

4. Breadth and depth of program— Incl ude in this statement any 
special degree requirements, requirements for courses outside the 
academic unit, the selection of a student's committee, the nature 
of the comprehensive examination and the dissertation. 
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5. Research in the program--Inclu<ie any information concerning how 
students are involved in the research, whether research 
assistantships are availaMe, how the research is funded, the 
emphasis upon research as a component^of the program. 

6. Public Service— Include specifically the interrelationships 
between public service activities and research and other aspects 
of the program, 

7. Teaching--Incr information concerning any innovations as well 
as assessments by students, faculty or alumni. 

Indicators of Program Quality 

It Evidence of effectiveness of the general education component of 
the curriculum (for undergraduates), including value-added 
calculation for the college using scores on the ACT CQMP exam~ 

2. Evidence of the achievement of desired outcomes of instruction for 
students, including results of comprehensive examinations and 
regional/national competitions, and documentation of placement and 
career success by graduates. 

3. Assessments of program quality by enrolled students (both majors 
and non^ma.iors) and graduates. 
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DEPAKTMENT OF RELIGIOUS STUDIES ~phonce^S^97V•2A^^ 

Dec. 11, 1981 

TO: Deans of the Colleges at UTK 

FROM: W, Lee Humphreys, 

Chair, Coordinating Committee on General Education 

Enclosed is an interim statement drawn up by the Coordinating 
Committee on General Education, When this committee was formed 
in late 1979, as a subcommittee of the Undergraduate Council, each 
college was asked to designate a general education committee of 
its o^'m. Our committee sent to the college committees a set of 
questions, and responses have been gathered, reviewed, discussed, 
and drawn together. The results have been tempered and supple- 
mented by the considered judgments of this committee. The en- 
closed statement is the result of that process. 

The committee now requests that you review this statement and 
pass on a copy to the person who chairs your college general 
education corrjriittee (if this committee has departed, disbanded, 
deceased, or defected, this is the chance for its renewal). 
We ask that you send to us your general observations about 
this statement as an ideal from which to consider the individual 
curricula of the colleges. We do not yet ask how this fits or 
is in line with your curriculum at this time or what specifically 
it would mean for them. . y 

The commit vee first would like to nrrise this statement as it 
feels warranted in the light of your general observations so 
that it can serve as a basis for consideration with each college 
of its curriculum. In this regard we would stress that the 
coordinating committee does not play a normative role, approving 
or designing programs. Our charge is to serve as a center for 
program assessment, to raise questions about general education 
from a university-wide perspective, and to gather and make 
available the resources and creative ideas of che several colleges. 
The goals of general education can be met in different ways, 
and the individual colleges are best situated to determine which 
finally meet their needs. At times we may, for necessary if 
lamentable reasons, fall short of our ideals. The committee 
wishes to bring its statement of ideals to the several curricula 
of the colleges in order to strengthen them where possible and to 
gather and make available the resources and constructive programs 
developed in distinct units at UTK 

We will be contacting the individual colleges for further meetings 
in the winter and spring quarters of 1982. To preserve our 
momentum and facilitate this work we ask that your initial obser- 
vations on the enclosed statement be sent to me by t^e end of the 
second week of winter quarter. 



ERIC 



Than;c you for your aid in this project, 

W. Lee Humphreys 
^ 509 McClung Tower 55 
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INTERIM REPORT OF THE COORDINATING COMMITTEE ON GENERAL EDUCATION 
DeccTiber 11, 1981 

The general education component of" all undergraduate curricula 
at UTK can be articulated across three distinct but interrelated 
areas: Basic Skills ; Knowledge; Judgment and Attitudes > 
In Sonne instances a specific course will focus on a subsection 
of one or more of these areas; in other cases areas will be 
honed and reinforced in courses who primary purpose may be to 
meet some other general educational need or to develop specialized 
knowledge and methods within a distinct discipline. Some of 
the items under "Judgment and Attitudes" cire reinforced in the 
larger extra-curricula life of students at UTK. 



BASIC SKILLS 

1. Verbal CorrjTiunicat ion in English 

A. All curricula require basic course work in English 
Composition (or demonstrated proficiency therein) . Several 
colleges also underscore the need to reinforce and further 
develop skill in written English in additional courses designed 
to meet other general educational needs and/or courses in a 
student's specialization. The committee is in agreement, noting 
that this skill, like most, will atrophy if not utilized. 
Effective writing should, therefore, be a stated requirement in 
a range of courses, and attention to its quality must serve as 
part of the basis for assessment and grading. Ideally, this 
should include work in a student's area of concentration in 
order to reinforce the fact that effective writing is essential 
not only for creative and responsible living in today's world/' 
but for success in one's profession or area of specialization. 

B. Skill in Spoken English is a second area noted by 
most colleges. In this regard actual practice varies. Some 
colleges require a course in speech, while others simply give 
lip service to this area in a general philosophical statement 
but do not implement it either in separate courses or as a 
stated component in other required courses . The committee be- 
lieves that in today's world facility with the spoken word is 
of such importance that attention to it must be a part of all 
undergraduate curricula, if not in speech courses then in 
courses with a stated recitation component. We would stress 
further that attention to ef f ect ive speaking should be 
accompanied by concentration on listening skills as an important 
part of oral communication. 

C. A few programs call attention to a need to stress 
RcaclincT Skills in the face of their perceived deterioration 
over the last decade. Little formal attention is given at 
present to this component in the curricula of the several 
colleges, and limited resources seem available for its imple- 
mentation. Ideally, students should enter UTK with the 
necessary reading skills. However, continued attention to 
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this perceived decline with some regular assessraent of students' 
reading skills is recommended^ for deterioration in this area 
undermines performance at all levels ♦ 

In regard to all three subheadings^ under ^'Verbal Communication 
in English" the committee urges that a minimal level for 
college entrance be defined, and that students admitted to 
UTK who fall below this level be required to take remedial 
work at the outset of their college experience. It is also 
important to assess the competence of transfer students in 
these skills. Introductory courses in written and spoken 
English must not be allowed gradually to deteriorate into re- 
medial programs whose standards and expectations fall below what 
we would wish to define as a basic level of college work. 

2. Computational Skills 

With the exception of the program in Fine Arts in the College 
of Liberal Arts (as well as the present (1972) curriculum in 
Liberal Arts) some work in Mathematics is required in all under- 
graduate curricula at UTK* The committee feels that a basic 
level of competence is desirable in Mathematics 
for all UTK graduates, and we wisk to raise the question of 
whether and to what extent this basic level should be the same 
as or beyond that attainable in a person's high school pre- 
paration. For many programs, of course,' a significant level 
of ability in some form of Mathematics beyond this base is 
necessary for later specialization. 

At present the entrance requirements in high school mathe- 
matics vary from college to college. In light of the recent 
changes in the university's admission patterns, in which a 
student is now first admitted to the university and only at a • 
second (and later) stage to a perticular college, the committee 
recommends the reassessment of all entrance requirements. The 
clear possibility of university-wide entrance requirements 
should be explored as a statement of a level of proficiency ex- 
pected for successful work in all undergraduate programs at UTK. 
It must be recognized that individuals who have not attained this 
level could well be admitted with a "deficiency" (an unfortunate 
choice of terms) and be required to take non-credit course work 
to develop the required skills (cn the pattern of mathematics 
and foreign language in che proposed curriculum for Liberal 
Arts. ) 

3 . Foreign Language 

At present only the College of Communication and the pro- 
posed curriculum in Liberal Arts require college-level work 
or demonstrated competence in a foreign language (there is an 
entrance requirement in foreign language in the present Liberal 
Arts curriculum) . Other programs acknowledge the value of this 
area of study, both for success in one's profession in a multi- 
na-ional world and for creative and responsible living in our 
glcoal environment. With this acknowledgement, however, it is 
noted that many programs are not able to include a foreign 



57 



language component as they are now designed. In liaht of this 
consideration a university -wide entrance requirement in high 
school foreign language should be considered (see item 2 above) . 

4 . Computer Skills 

Several colleges drew attention to the importance of 
acquiring basic knowledge and skill in the operation and utili- 
It in today's world. This committee suggests 
that this area be assessed along with foreign language because 
of Its importance in the development of basic communication 
skills. We suggest that work in either foreign language or 
computer skills should be a part of all undergraduate curricula. 
It IS noteworthy that the programs finding the greatest diffi- 
culty with a foreign language requirement often stress the 
need .or skill in the use of computers and understanding their 
potential. ^ 

5. Problem Analysis and Solving 

The formation of precise questions, the analysis of types 
of reasoning, the assessment of evidence, and the ability to 
critique an argument and weigh the claims of others are all 
fundamental characteristics of a creative and disciplined 
mma Several colleges stressed the value of developing facility 
in these characteristics as a vital part of general education, 
not only for success in an area of specialization but also for 
responsible living. While it is possible to attend to these 
SKills in one or more courses in inductive or deductive logic. 
It IS often more desirable that a full range of courses address 
implicit methodological questions. Whether attained in a specific 
course or as an aspect of other courses, these skills must be 
consciously utilized and developed throughout one's college " " 
program. ^ 

Kh'OWLCDGE 

As we move to the second and third areas unanimity among 
colleges IS not always as pronounced, what follows represents 
l]^t^nTT ^ judgments regarding areas receiving substantial 

notice from several of the colleges. These areas may be ap- 
proached fro:;^ a wide range of perspectives that cut across 
diotmct colleges and programs, and it is clearly possible 
tha. more than one can be taken up in a given course or se- 
quence of courses. It is not the intention of the committee 
nro^^ni'''?'''' or design specific courses. The areas defined are 
presented as vital for all educated persons, and serve not simply 
as a basis .or more in-depth work in a discipline or profession. 

1 . Aosthot ic3 

The several colleges generally agreed that appreciation of 
ror.-3 or tne arts is vital to any program of general education. 
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Attention to literature, art, music, architecture, and the 
theatre as modes of human expression and creativity and as 
attempts to articulate through symbols human experience and a 
sense of place in the cosmos should be a part of the back- 
ground of all who receive a baccalaureate degree from UTK. This 
experience may be attained either through courses that stress 
the practice of some form of the arts or through their appreci- 
ation. It is not, however, enough simply to ask students to 
select froni an ill-focused list of humanities courses, and it 
must be recognized that attention to the aesthetic dimensions of 
human life is appropraite in a wide range of programs and 
courses. 



2. The Scientific Basis for Life 

The committee wishes to include within this broad designa- 
tion attention to human wellness, to the basis of life and the 
functions of the human body in its environment that lead to a 
state of relative well-being. In this regard nutrition and 
human development are considered essential. Respect for the 
environment and an understanding of ecological processes and 
balances are fundamental as well. There was near unanimity 
among colleges that attention to the basic methods of 
science and the processes of scientific inquiry and argu- 
mentation also form a necessary component of an educated per- 
son's perspective. 



3 . Technology 

Technology, in the broad sense, is an inherent part of human 
endeavor and, as an interaction of human skills, science, en- 
gineering, economics, and invention provides for the needs arid * 
desires of individuals and their societies. Technology does and 
will increasingly use physical and human resources. The techno- 
logical perspective implies an understanding of these inter- 
actions independent of an individual's discipline or profession. 
All undergraduates must understand that the forces and dynamics 
of technological change impact lives in many ways, and ignorance 
of them limits the ability to direct change wisely. 



4 . History of the Western World 

We live in time and are informed fundamentally by our past. 
Therefore, attention to the major forces, movements, institutions, 
and persons in our western heritage is generally recognized as 
essential for sel f -understanding by the several colleges. Con- 
sideration of the methods utilized by the historial in assess- 
ment of evidence and construction of arguments is also im- 
portant, as is attention to the basic cultural, economic, 
political, social, and other factors that influence human 
chanTG through time. 
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5. Foreign Culture 



We live in a world that now binds cultures and nations to- 
gether in ways that demand immediate and intense contact. Atten- 
tion to distinct cultural traditions, to different literatures 
and arts, to distinct patterns of value, different political 
and legal institutions, philosophical and religious traditions, 
as well as histories and senses of history, expose us to enduring 
patterns of civilization that are not immediately our own and 
provide us with a distinct perspective regarding our own. 
Attention to the problems and possibilities for communication be- 
tween cultural traditions is an important component in this. 



6. Economics and the Management of Resources 

Several colleges directed attention to the world-wide 
economic factors that define human life, including a basic 
understanding of the dynamics of the market and the production 
and distribution of essential goods and services. Attention 
should also be given to the issues that face us as consumers as 
well as producers, and to the articulation of values and 
priorities expressed through the ways as individuals and as 
groups and nations we handle and account for our handling of 
resources and skills. The complexity of and competition 
between values and priorities, often on a multi-national 
level, demands consideration. 



7. The Social Sciences 



Understanding of the social forces at work in the world, 
including especially the changing roles of education, business, 
government, politics and the law, the media, and the family is • 
a vital component of general education that each college noted 
in some manner. The methods and perspectives of the social 
sciences impact on many programs and courses in the several 
colleges and divisions that make up UTK. Attention to this 
should include exposure to the origins, nature, and some evalua- 
tion of the scientific and humanistic approaches to human social 
life and the individual's development of a sense of self. A 
basic facility in the social scientific presentation of data 
and the interpretation of statistics should be attained as well. 



III. ATTITUDES a:>;d judgments 

Within this broad category the committee has drawn together 
a range of qualities of mind and action that were stressed by 
several colleges that we believe can be reinforced and honed 
m both the classroom and the extra-curricula life at UTK. 
Utilization or the nany resources and opportunities across this 
carpus that are not directly linked to the classroom, ranging 
i.rom visiting speakers, theatrical production, concerts, dis- 
plays, the activities of university governance and program 
development, etc., through loss formal aspects of student 

Er|c 6o 
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life, provides resources for adding richness and a dimension of 
experience to the formal curricula. The awareness that a student's 
development in his/her years at UTK are strongly influenced by 
formal and informal structures of life on this campus is im- 
portant in the assessment of the impact of any curriculum. 

1. Values 

The ability to articulate value questions, to seek necessary 
information, to imagine creative alternatives, to assess solutions, 
and to be aware of the immediate and indirect imp?.ications of 
decisions and actions is essential for responsible life in 
today's world. Awareness of the ways in which specialized areas 
of knowledge and activity shape the lives of others and the 
environment is an aspect of professional life that cannot be 
ignored. Exposure to those norms and issues that have traditionally 
informed our value decisions as individuals and as groups should 
accompany self-reflective consideration of one's personal values. 
This is not to be so construed that the university is perceived 
as imparting a particular code of conduct or sectarian set of 
values beyond those essential to the search for truth, re- 
sponsible citizenship, a sensitivity to competing values, and 
understanding of different positions. 

2 . The Dynamics of the Political and Social Arenas 

The commit^.ee deems it important that the t,he student be ex- 
posed to the process of working with others in setting goals, 
constructing programs of action to meet and implement goals, and 
assessing a course of action undertaken. Awareness of the 
structures of control within a society and the limits in v/hich 
leadership is exercised and responsible social action initiated 
is a component of this. Vaules are often implemented through 
formal and informal political and social groups. The ability 
to understand the processes of such groups and, if desired, to 
work effectively within them is important. 

3. Personal Wholeness 

The committee supports the belief expressed by some colleges 
that a vital aspect of one's college experience should be the 
opportunity and challenge to reflect personally on one's 
goals and values as well as one's opportunities. An appropraite 
integration of self-knowledge, abilities, interests, and values 
with current career opportunities leads to responsible and 
creative lives and personal satisfaction. 

4 . Life - Long Loarning 

Within all specializations and professions new skills and 
knov;Lodae will repeatedly have to be mastered, for knowledge 
is -^.ot static and is never complete. Many will be asked to 
re:::ol, to take on new challenges and to seize new opportunities. 
One's college experience should open one onto a life-long 
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course of learning and provide a cutting edge that will 
facilitate continued individual and professional development 
and the ability to live creatively with change, 

5 ♦ Experience in Learning 

It is no denial of the vital place of the classroom and lab 
to observe that the world is not simply a set of classrooms and 
labs. An experiential dimension of the relationship of learning 
to the larger world is desirable where appropraite and possible. 
This could range from formal programs of field work and co- 
oking through close working with others, both faculty and student 
on a creative project in one's area of interest. The nature 
of such work will, of course, vary greatly from program to 
program. 



Coordinating CorCTittee on General Education: 

W. Lee Humphreys, Chair 

Religious Studies 
Lawrence M. DeRidder 

Educational Psychology 
Thomas C. Hood 

Sociology 
Durward S. Jones 

Lav; 
Xen Kenney 

Graduate School of Planning 
Roy F. Knight 

Architecture 
Kelly Leiter 

Journalism 
Richard C. Reizenstein 

Marketing and Transportation 
Jane Savage 

FSNFSA 
Laurence N. Skold 

Plant and Soil Science 
E. Eugene Stansbury 

Chemical, Metallurgical, and Polymer Engineering 
Otis H. Stephens 

Political Science 
Roy Smith 

Student Member 
Andy Hoover 

Student Member 
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Interview with Dean Bill Cof field 
College of Education 
August 1981 



Specific Field Test 

National Teacher Exam required of all seniors who are prepared to teach. Data 
from 3-4 years (NTE is not a valid predictor of teaching effectiveness,) NTE will 
change dramatically next year. No use made of results because they are not curriculum- 
related. 

General Education 

In 1980 36 seniors in Education took the ACT COMP. They represented the College 
fairly. In 1981 only 4 seniors showed up for the exam. 

No use was made of 1980 scores on COMP. 

Student Satisfaction 

1) Quarterly assessment of the teacher ed program is provided by students com- 
pleting student teaching. 

Use: Results are sent to each department head each quarter. Consistently low 
marks for preparation in learning theory, classroom management, and tests and 
measurement has lead Dean to call for curriculum change in the Ed and Counseling 
Psychology Department. 

2) BERS conducts each year a survey of a sample of graduates asking for 
assessments of the teacher education program. 

3) In 1981 a survey form was sent to principals who employ College of Ed graduates 
Findings from (2) and (3) are made available to department heads. 

4) Program Committee 4 years ago developed an evaluation scheme for use in all 
program areas. 

Use; Department of Curriculum and Instruction dropped 95 courses and added 42 
new ones. This kind of evaluation will be conducted again in 1982-83. 

5) Evaluation of classroom teaching - Every department employs some form of 
student feedback. Annual review by department head and applications for promotion 
require some "objective evidence" of teaching effectiveness. Approximately 80% of 
faculty use an evaluation form (Dean's estimate) ranging from a few questions to 
Milton's form. Dean feels peer evaluation would be worthless — no one will speak 
ill of a colleague. 

6) No college-bycollege report was provided for colleges using the ACT Student 
Opinion Survey. Thus no use was made of the results. 

7) Coffield wants to institute exit interviews with graduating seniors. 

8) Deaf education graduates used to be asked to complete a survey evaluating 
their undergraduate program. But this was discontinued. Now no department follows 
up its graduates. 



Evaluation consultants; Tom George and Sky Huck 
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REPORT 

_ _ OF THE 

SiUDtN-T OUTCOMES TASK FORCE ON GENERAL EDUCATION 



General Education at UTK 
In order to Identify and/or construct valid method, for .ea.urln, .tudont 
.cKleve»e„t 1„ general education at the University of Tenneaaee. Kno^lUe. It 
waa eaaentlal first to define the goals of general education at UTK. The TasU 
Force on General Education adopted as Its wording statement of these goals the 
October n, 1,8. Report of the UTK Coordinating Co^ittee on Ceneral Education. 

That report contains the recommend • 'on fh=f ,1,. , . 

on that the general education coniponent of 

all undergraduate curricula at UTK be articulated across three distinct but 
interrelated areas: Basic SWls. Kno„ledge, Judgments and Attitudes. An 
outline of the contents of the three general areas appears below. 
I. Basic Skills 

1. Verbal Communication in English 

A. English Composition 

B. Spoken English 

C. Reading Skills 

2. Computational Skills 

3. Foreign Language 

4. Computer Skills 

5- Problem Analysis and Solving 
II. Knowledge 

1. Aesthetics 

2. The Scientific Basis for Life 
Technology 

4. History of the Western World 

5. Foreign Cultura 

6. Economics and the Management of Resources 

7. The Social Sciences 
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III* Attitudes and Judgments 
!♦ Values 

2. The Dynamics of Political and Social Arenas 

3. Personal Wholeness 

4. Life-long Learning 

5* Experience in Learning 
The Report of the Coordinating Committee on General Education is a draft 
of a statement of broad goals for general education at UTK. Selection of a 
methodology for measurement of student achievement should be 'jased on a state- 
ment of much greater specificity ~ that of student objectives or competencies 
related to each of the goals of general education.. However, the goals statement 
was accepted because it was the best guide to faculty thinking about general 
education that was available on the campus in 1982, and because the study of the 
field of measurement of general education which was undertaken by members of the 
Task Force revealed that the state-of-the-art in specificity of statements of 
desirable student outcomes at UTK is paralleled by the state-of-the-art in measure- 
ment of general education outcomes at the national level. 

Measures of General Education Outcomes 
The Task Force membership considered four of the most widely known current 
standardized methodologies for measurement of general education outcomes: 

1) The ACT COMP (£omprehensive Outcome Measures Project) 

2) The ETS Undergraduate Assessment program 

3) Georgia's Regents' Examination 

A) A series of instruments developed by McBer and Company. 
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Items 2 and 3 above were eliminated rather quickly from detailed consideration 
because -the UAP was discontinued by eTS in 1981 and because Georgia's Regents' 
Exam is a test of communication skills alone. 

The ACT COMP is a test of the competencies or outcomes which should result 
from general education aa defined by a representative group of educators in 
some 150 postsecondary institutions and agencies. The test purports to 
measure skills necessary for effective adult functioning. Its development 
began in 1976 and it has been administered three times (Spring 1980, 1981 and 
1932) to samples of UTK graduating seniors. Six scale scores may be obtained [ 
from the COMP: 1 

1. Communicating 4. Functioning Within Social Institutions - 

2. Solving Problems 5. using Science and Technology 

3. Clarifying Values 6. Using the Arts 

Members of the Task Force on General Education concluded that the COMP 
measured reasonably well in a general way student achievement in "Basic Skills" 
and "Knowledge." UTK General Education Areas I and II. However, the specific 
areas of Computational Skills. Foreign Language. Computer Skills. History of 
the Western World, and Foreign Culture are not addressed to a satisfactory extent. 
In the COMP Objective Test, the multiple choice form of the exam that has been ,7,ivea 
heretofore at UTK. Verbal Communication in English is not adequately assessed. 
However, the longer form, the Composite Examination, does contain more satis- 
factory means of assessing Verbal Communication skills . 

Three of the McBer instruments were studied by Task Force members: ' 

1) Test of Thematic Analysis 

2) Analysis of Argument Test 

3) Thematic Apperception Test 
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Although there are deficiencies in reliability and validity, scoring of 
protcccls is relatively expensive, and the tects do not adequately measure basic 
skills or knowledge, the McBer instruments do measure some aspects of the 
concepts included in UTK General Education Area III, "Attitudes and Judgments." 
Such individual characteristics as motivation, intellectual flexibility, and self- 
definition are measured, as are thought patterns, attitudes, and analytical 
capacities. Students being tested are required to analyze, evaluate, and synthesize 
data; to communicate their thoughts and defend their arguments. 
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Task Force Recommendations 

Members of the Task Force on General Education endorsed unanimously the 
concept of testing students' achievement of specified general education out- 
comes. Increasingly the University is called upon to demonstrate the quality 
of its programs and the posiMve impact of these programs on students. Periodic 
administrations at UTK of one or more measures of achievement in general 
education can provide a systematic basis for assessing : 

1) the extent to which the general education objectives specified by the 
institution are being achieved by its students; 

2) the level of achievement of general education objectives by students 
at UTK as compared with students at other post-secondary institutions, 
i.e., as compared with national norms; and 

3) the relative effects of significant curriculum changes undertaken 
by the institution. 

The information thus obtained can be used to demonstrate the effective- 
ness of the general education program at UTK to students and their parents; 
to alumni and friends of the University; to the public; and to the Tennessee 
Higher Education Commission, whose performance funding mechanism requires 
such evidence. The information also can be used in a very practical way by 
faculty and administrators in decision-making and planning for program improve- 
ment ; e.g., identification of ^trong and weak program components has implica- 
tions for: 

1) evaluating teaching effectiveness; 

2) adding, modifying, or phasing out courses or program areas; 

3) budgeting; and 

4) selecting materials for the library. 
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On the basis of its study of measurement methodologies the Task Force 
concluded that no examination which is available curtently will assess adequately 
student achievement of all general education competencies considered important 
by curriculum planners at UTK* However, despite its limitations, the ACT COMP 
is the best instrument available now for measuring genera] education outcomes 
at this institution. Basic skills and general knowledge are assessed in a 
competent manner; the examinee receives information in a variety of modes: 
by reading, listening, and viewing; and the format, length, and structure of 
the COMP make it relatively economical ($8.00 per scored student protocol) 
to administer and score. It should be noted that the ACT COMP is designed to 
produce group data that can be used in program assessment and improvement. 
Individual scores are not sufficiently reliable to make the instrument appro- 
priate for advising students concerning their own strengths and limitations. 

Task Force members perceived the NcBer Test of Thematic Analysis and 
Analysis of Argument Test to be promising instruments for the measurement of * 
certain characteristics of self. For this reason the group endorsed exploratory 
work at UTK with these McBer instruments. 

In light of the foregoing, the Task Force on General Education recommends 
the following actions: 

1) The ACT COMP Objective Test should be given annually, at University 
expense, to all incoming freshmen and to all graduating seniors so that 
the comparison of pre- and post-program scores may yield a measure of 
value added as a result of the educational experience provided at UTK. 
In light of the costs involved in scoring the test, a representative 
sample of answer sheets may be selected for processing by ACT. 

2) A statement of the requirement for all freshmen and all seniors to take 
a test which measures achievement in general education should be placed in 
the catalog of the University of Tennessee, Knoxville. 
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3) Programs should be conducted for the purpose of acquainting a large 
segment of the faculty with the objectives and potential value to the 
institution of measuring achievement in general education. 
A) All current general education testing methodologies should be considered 
tentative and in a state of evolution. Colleges should be encouraged to 
develop supplementary measures of their own, such as follow-up studies of 
graduates* perceptions of own achievement. An interdisciplinary group of 
measurement specialists at UTK should be identified and charged with the 
responsibilities of: 

a. assessing the usefulness of the ACT Objective Test and Composite 
Examination (the long form of the test which requires the examinee 
to construct some written responses) and the Activities Inventory 
for the measurement of general education outcomes at UTK; 

b. assessing the usefulness of the McBer tests as a supplement to 
the COMP in measuring characteristics of self; and 

c. constructing valid methods of measuring those outcomes of general 
education at UTK which are not measured by the ACT or McBer 
instruments, such as computational and computer skills and cultural 
perspectives. 
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n ry r\ n fv 

of the 

STUDENT OUTCOMES TASK FORCE ON STUDENT ACHIEVEMENT 
IN SPECIFIC FIELDS 

Sources of Information Used Most Often in Assessment of Academic Programs 

In its initial deliberations the Student Outcomes Task Force on Achievement in 
Specific Fields questioned the relative importance of student outcomes data as a 
source of information for assessing and improving academic programs. 

Task Force members acknowledged that student scores on standardized instru- 
ments such as the Fundamentals of Engineering Examination and the National 
Teacher Examination, and evaluations of student performance by faculty and/or 
employers, constituted comparatively objective information for use in program 
assessment and decision-making. Nevertheless, members were able to provide 
documentation for their contention that several other information sources, 
some much less objective, were used more often and given more weight in decision- 
making than student outcomes information. The information sources identified 
included: 

1) Faculty 

a. Formal and informal ass<'*ssments and recommendations 

b. Committee recommendations 

2) Administrators 

a. Philosophical/managerial orientation of department head or dean 

b. Directives from central administrators 

c. Institutional goals and designated responsibilities 
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3) Students 

a. Formal and informal feedback 

b. Structured contacts with selected students (e»g., Deans' advisory group 
c* Surveys of current students and alumni to obtain their perceptions 

of their achievement 

4) Accrediting organizations 

a. Specified program criteria 

b. Local unit goals prepared in response to general guidelines 
c* Self-study data prepared for an accreditation review 

d. Reviewers' recommendations following an accreditation review 

5) Employers 

a. Informal contacts with employers at professional meetings, social 
occasions 

b. Remarks made by employers concerning the performance of student interns 

6) Mandates provided by federal and state laws 

7) Mandates provided by policies of state licensing or governing boards 

8) Professional associations and other interest groups 

Members of the Task Force contended that most University faculty and adminis- 
trators are quite comfortable using the relatively inexpensive, easy-to-obtain 
information from these varied sources awd have some doubt that the quality or 
substance of their decisions about academic programs would be altered or improved 
significantly by the use of structured data obtained from more expensive 
measures of student achievement. At least four reasons for this doubt were 
identified: 
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1) A test of student achievement in a specific field merely samples the skills 
and knowledge imparted by the educational experience in the field. 

2) The types of questions and scoring procedures used ^n such tests often leave ^ 
unanswered questions about what the student actually knows. For example, 
multiple choice questions do not permit analysis of the thought processes 
in which the student engages to arrive at a response. Partial credit Is 
not given for a nearly correct response, and on the other hand, guessing 
is not penalized. 

3) Unless all seniors in a program take the advanced test in their field, 
questions are raised about the representativeness of the sample that did 
take the test. 

4) Unless the advanced examination in a specific field is given to freshme;i, 
which in most cases is clearly inappropriate, there are no baseline data 
against which to compare senior scores and thus to evaluate the value 
added by the educational experience. 

Nonetheless, some Task Force members voiced strong support for the systematic 
collection of structured information about student achievement in their major 
field of study because: 

1) If an examination can be located or constructed that measures some or all 
of the objectives of a course of study, student scores on such an examination 
can constitute valid evidence of program effectiveness and suggest areas 
for improvement. 

2) Standardized test scores constitute normative data that permit comparison 
of student achievement across programs or institutions. If students in a 
gi.en program consistently achieve high scores. the program's prestige is 
enhanced . 
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J^ources of Inforn^ation Ccr.cerr.ir.g Student Achievement in Specific Fields 

Task Force members considered each of the measurer^ of student outcomes 
in specific fields that is available for programs offered at UTK. In so 
doing they noted the difficulty inherent in attempting to distinguish achieve- 
ment in some major fields from achievement in general education. Seven types 
of specific field measures were identified: 

1) Standardized examination (licensing exams In nursing, engineering, 
architecture, the National Teacher Exam, etc.) 

2) Locally developed comprehensive examination or competency assessment 
based on objectives established by the academic unit. 

3) Evaluation by faculty of a comprehensive student achievement', for exampl 
observation of a student performing a task; assessment of a work of art, 
a design portfolio, a technical paper. 

A) Evaluation of a comprehensive student achievement by an external 
reviewer, i.e., member of a visiting accreditation or program 
review team, an internship supervisor, a co-op or other employer. 

5) End-of-program assessment by seniors, reporting perceptions of 
their own achievement (may be written or obtained in an exit 
interview) . 

6) Retrospective assessment of their achievement by alumni one or 
several years after graduation. 

7) Assessment by empl<^yers of the competencies of alumni one or 
several years after graduation. 

The following examples of the above measures are included in Appendix A: 

Example of ill: Specification of competencies for State Certifi- 
cation Area 4 for school psychologists 

Example of //j: Evaluation of Student Teaching 

Example of //A: Praccicum Supervisor Evaluation 

Example of //5: Education: Student Teachers Evaluation of the 
Teacher Education Program 

Example of Survey of UTK College of Engineering Graduates 



Example of ill: 



College 



of Nursing employer questionnaire 
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Analysis of Measures of Student Outcomes in Specific Fields 

Several of the academic units that employ a standardized measure of achieve- 
ment in the major field of study for seniors or graduates (Measure //I above) 
completed the form "Assessment of Advanced Examinations in Specific Fields" 
(see Appendix B). The results are recorded in Table 1. 

In general, the costs of administering and scoring a standardized examination 
that is required for licensure are borne by the student rather than the institution. 
For instance, the Engineering student who takes the Fundamentals of Engineering 
exam pays a $15 fee, and a charge of $50 is paid by each student who takes the 
State Board Exam in Nursing. The Business Assessment Test, which is not a formal 
requirement for graduation, has been administered by the College of Business at 
a cost of $1800 per year. 



Several academic units at UTK reported that there is no standardized instru- 
ment for measuring student achievement in their programs. 

Representatives of two of these units responded to a suggestion in the form 
"Locally Developed Field Test" (see Appendix B) that they develop their own 
competency-based assessment of student achievement (Measure )f2 above). Their 
responses suggested that most units would consider it difficult, if not impossible, 
and perhaps philosophically unacceptable, to develop such a measure. Reasons given 
for this position include: 

1) difficulty in achieving agreement among faculty on a specific statement of 
measurable program goals and objectives; 

2) time and cost involved in constructing a reliable, valid assessment 
instrument; 

3) limitations in locating persons with sufficient technical training and 
experience to undertake the task of test development. 
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TABLE 1 

Some Information About Advanced Examinations In Specific Fields 



Name of Exam 



Is Total Score 
available for 
^' gur graduates? 



How is 
Total Score 
. Used? 



Are 

subtest scores 
available for 
your graduates? 



National 

Teacher 

Examinations 



Hov are 
subtest scores 
used? 



Reason (s) for 
Giving Exam 



Problems with 
Making Use of Scores 



Yes 



Not used. No 
cut-off score 
established. 
Test being 
revised. 



Yes 



Not used. 

No cut-off scores 

established . 



State Boacd Exam has been required 

Requirement for too short a f^me for 

meaningful resumes to be 

available. 



State Board 
of Nursing 
Licensure 
Examination 



Yes 



As one source of 
Information for 
making decisions 
about effective- 
ness of college 
curriculum. 



No 



Graduates need the 
license to practice 
nursing. 



None 



Fundamentals of 
Engineering Exam 



Yes, by major 
but not for each 
individual . 



As one source of 
Information about 
curriculum effec- 
t Iveness . However , 
Information is not 
course specific. 



No 



It constitutes the 
first step toward 
legal recognition 
for practice of 
engineering. 



Business 

Assessment 

Test 



Only fundamental knowledge 
and skills are measured, 
some of which are acquired 
in course work outside 
College of Engineering. 
Exam does not measure 
learning in specific 
engineering fields. 



Yes 



00 



In support of 
continuing accredi- 
tation and in demon- 
strating the quality 
of the College 
curriculum. 



Yes 




As one source of 
information for 
assessing strengths 
and weaknesses of 
curriculum. Changes 
have been made as a 
result of applying 
information gained 
from the test scores. 



To obtain macro 
assessments of 
curriculum effec- 
tiveness. To obtain 
relatively objective 
measure of program 
effectiveness that 
can be used in demon- 
strating program 
quality to alumni » 
accrediting agencies, 
.My«rsity,od«lniitrctor»« 



Test no longer available. 
Some formalization of the 
right of the College to 
require such an examination 
would be helpful. 
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Measures //3-//7, while somev;hat more subjective than //I and //2, appear to 
enjoy more widespread acceptance among representatives of academic units at UTK. 
Design of the evaluative criteria used in these measures does not require the 
level of expertise in measurement that is needed to construct a standardized 
paper-pencil examination. Thus individual faculty in the academic unit can be 
involved in developing, as well as administering, and scoring or reviewing. 
Measures //S-//?. The cost of these procedures is usually borne by the unit* 
Exclusive of the faculty time required to design the instrument and interpret 
the results, the cost bf associated with Measures '/S-//? varies fron a modest 
figure for duplicating sufficient quantities of the instrument to approximately 
$500 for a follow up survey involving multiple mailings to several hundred 
aiumni* 
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RECOMMENDATIONS 

On the basis of its study of measures of achievement in specific fields, 
the Task Force makes the following recommendations: 

1* Faculty in each academic unit should determine whether or not there is 
a standardized instrument available for measuring student achievement in the major 
field(s) of study offered by the unit. Inquiries should be made through professional 
organizations, accrediting or licensing agencies, and/or research/service units 
at UTK. If an acceptable standardized instrument is available, the academic unit 
should require that every graduating senior take the examination. 

Mean scores for classes of students should be compared with 
national means in making judgments concerning prograa quality. 

2. If the academic unit does not have access to a standardized test of student 
achievement in the major field of study, the faculty should consider undertaking 
the task of developing its own measure. Since the development of a reliable and 
valid standardized test requires considerable time, expense, and technical expertise, 
caution is advised in choosing this approach. 

3. As alternatives, and as supplements, to standardized tests, faculty in 
each academic unit should develop or adapt one or more of the following metho- 
dologies for systematically gathering data on student achievement in the major 
field of study (see page 4 for more detailed explanation of each): 



a. Evaluation by faculty of a comprehensive student achievement 

b. Evaluation of a comprehensive student achievement by external reviewer 

c. End-of-prograra assessment by seniors, reporting perceptions of their 
own achievement 
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d. Retrospective assessment of their achicvcir.Gnt by alumni. 

e. Assessment by employers of the competencies of alumni one or severaj 
years after graduation* 

These methodologies should be employed annually if economical to do so, or 
at 2- to 5-year intervals as the unit deems appropriate. The timing should be 
such that the information is s Jficiently current and representative co be useful In ^ 
program planning and decision-making. In addition to the immediate utility to the 
academic unit in demonstrating program quality to various audiences and in 
identifying areas from improvement, data thus collected can be used in comprehensive 
program reviews and in reporting to accrediting agencies and the Tennessee Higher 
Education Commission. 



Area 4 

State Certification Area Competency Definition 

"The psychologist should be able to devise strateyies for assisting the 
school administration and staff to evaluate the effectiveness of the 
school program in meeting the academic and social needs of the students 
and adults the school serves. 

Specifically, the school psychologist should be able to develop and 
eva uate a wide variety of data-gathering investigations; to understand 
basic inferential and descriptive statistics; to gather and interpret 
DrJaJai .'n.'J^ ^^^^fj^^ ^^^^cts psychological and educational 

programs, and to translate research data into psycheducational practice " 



4.1 Demonstrates knowledge of basic descriptive and inferential statistical concepts 
and methods including: 

4.11 Descriptive statistical methods incluoing properties of raw and various 
converted scores, frequency distributions S their graphical representations, 
measures of central tendency, & measures of variability. 

4.12 Inferential statistical methods including probability theory, t-test, 
ANOVA, ond 

non-paramstrics* 

4.13 Correlational methods to analyze reliability, validity, and association 
between variables. 

4.2 Demonstrates knowledge of research and evaluation as applied in field settings. 

4.21 Given a significant school-related problem, locates and reviews relevant 
li terature. 

4.22 Selects an appropriate investigation strategy and plans for collection 
of relevant data. 

4.23 Demonstrates a functional knowledge of contemporary research standards, 
especially with respect to ethical safeguards and legal requirements. 

4.24 Interprets and con.nunicates research information in a manner that is 
intelligible to and useable by teachers, administrators, students, 
school boards, or parent groups. 

^4.25 Demonstrates basic user knowledge'of computer system capabilities. 
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3 Collaborates with administrators and others in reviewing and evaluating a 
system-wide testing and evaluation program. 



Area 5 

State Certification Area Competency Definition 

"The psychologist should demonstrate ability to understand the roles 
of others with whom he will have to work in the school & comnunity. 

Specifically, the psychologist should demonstrate knowledge of the 
tor. t»r^E including goals, of other personnel; e.g. administra- 
tors, teachers, social workers, health workers, attendance workers, 
guidance personnel, and special education workers. He/she should 

cim^T ""^^^ his ability to identify and work with these and other 
commumty resources. 



5.1 Discriminates between and describes the stated (formal) and implicit (informal) 
goals and role functions performed by administrators, teachers, a.-^d support 
personnel (including self). 

5.2 Identifies the typical referral agencies in a metropolitan area and describes 
their roles and functions. 

5.3 Serves as an efficient linkage between client(s) and service(s) by selecting 
and utilizing resources suited to the particular needs of an individual or 
group. 



Area 6 

State Certification Area Competency Definition 

"The psychologist should demonstrate knowledge of the organizational 

other sv'^t.'!^ ?r;r'" °' their'inter-relationship w? h 

?tr.[pn?LT ^ommny. He should demonstrate knowledge of 

vLoJy^jrrsonn'e? ''"^ ' functioning of administrative and super- 



' rHcilarn.tJLn^^5^°-^ demonstrate familiarity with the 

■mova? nn.'^Jn ^h?! ^yP^^^UL'^'' '^^^^ as Well as with 

..movations in this area. This ma.v include such things as knowledge ot^ 
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EVALUATION OF STUDENT TEACHING 



The Universily of Tennessee/Knoxvilie 



College of Education 



Name of student teacher 



Last 



First 



Gude: 



Satisfactory 



Middle 



No Credit 



University Coordinator , 



Cooperating Tcachcr{s) . 



Student Teaching Center 



Subject area and/or grade level in which student teaching was completed 



Quarter , 



Year 



STUDENT RELEASE REQUEST 
I request that this evaluation become a part of my placement record. 



Signature 



I hereby grant permission to the College of Education, University of Ten 
evaluation. 



nesscc, to extend to prospective cmplo>crs ihe contents of this 
Signature 



DIRECTIONS FOR EVALUATING 
Following are studenfteacher characteristics believed to have an influence on teaching effectiveness. The cont.--..'^^ offers a means 
Lt?of^'4o^^^^^^^ ^^^r.c.n.'. can be compared to other student eeachers-nof^ en eJ a^^^^^^^^^ 

A rat ng of Good on th.s scale ind.cates that the characteristic being considered is zcT^.tiHiTd ihlTlhTstudent te.cner will probably 

the student teacher will be c.ther more or less effective than "Good" for that particular characteristic in a normal classroo.-n SKuatTon 

ic InforL^^i-^r^ T'"" ^^PPonlm evidence of the rating on the continuum. These comments aay provide specif. 

rating S ' '""''^ "^"'^'""^ "^^"^^'^ b« presented on a 



SUPERIOR: 

Reveals qualities found 
in only the most effec- 
tive and creative stu- 
dent teachers. 



VERY GOOD: 

Clearly above average. 
Goes well beyond meet- 
ing the basic require* 
ments. 



GOOD: 

Acceptable, respectable 
performance. Meets ba* 
sic requirements. 



MARGINAL: 

Acceptability only mar- 
ginal; the quality of the 
characteristic needs im- 
provement. 



UNACCEPTABLE: 

Quality of the charac. 
teristic is below mini- 
mum standards for cer- 
tification. 



SUPERIOR: 

L L 



VHRY GOOD: GOOD: 



i. App«)fent health .md vii.jlitv 



J L 



MARGINAL: 
I 



Comment: 



3. Voice and language usage: 



Comment: 



4 Ab.l.iy to work cooperalively w.ih olher staff members: 




Comment: 



S. Ab.lHy »o profit f,om feedback supplied by obse,vai,onal lechmques: 

I I I 



Comment: 



UNACCEPTABLE 





6. Ability to make reahsltc decisions, has maturity of judgment: 

i III! 

Comment: 



PRACTICUM SUPERVISOR EVALUATION 
Practicum Student ^^^^^^^^ 

1. Kow many hours did the student work each week? Total hours for the term? 

^' flrm^^^oSI'J^''"'" '^"S"^ ^""^ ^° schedule as outlined at beginning of school 
term, report! ng^promptly and regularly throughout the period covered? 

3. Briefly, summarize various responsibilities and tasks performed while assigned to you. 




4. Was the student fast enough in the performance of assigned work? Yss 



No 



sit'isfacto'^yr'iL'!!!;;'' worU.abits appear to be good and was overall performance 
a^^ '^S.^r^'st^^l^^^^^^^^ ^^^^ -11 With associate! 

fo^l ^^i:^^^^^'^ -"^-t appear j 

8. How would you rate the student's overall performance? Excellent Gocd ' 5 

Average Below Average 

9. Are there areas in which the student needs to improve? Yes 
If yes, which ones? 



No 



10. If you had a vacancy on your staff, would you consider employinq the student nn ? 
permanent basis in a position for which he/she were qua? if?ed? Yel No 
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REMARKS (Please give question number and elaborate upon answers ?bove as necessafv 
Do you have suggestions for improving the practicum work experiencrDroaram S 
educational background or qualifications of students in the'^p^og^am'r 



(Date) 



S9 



(supervisor signature) 



TtitTef 



STUDENT TEACHERS' EVALUATION OF THE TEACHER EDUCATION PROGRAM 

(Expressed in Percent) 
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1 

} 




1975- 
1976 


1930- 
1981 


1975- 
1976 


1980- 
1981 


1975- 
1976 


1980- 
1981 


1975- 
1976 


1980- 
1981 


1 
\ 


1. The degree to which the first 90 hours 
(General Education) contributed to your 
development as an "educated person" was: 


11.2 


13.6 


55.8 


64.4 


22.5 


17.8 


4.1 


1.7 


I 


2. Your insight into your own philosophy 
of education i^ now: 


28.6 


25.7 


64.1 


67.7 


2.5 


4.3 


0.7 


0.7 




3. Your understanding of the social 
bases is now: 


16.9 


16.2 


67.1 


70.0 


11.0 


11.2 


1.0 


0.4 




4. Your understanding of the historical 
background to contemporary issues is now: 


6.3 


6.5 


55.9 


63.8 


28.3 


25.7 


4.9 


2.4 




5. Your required class observations 
prior to student teaching were: 


15.8 


17.2 


40.5 


47.0 


29.0 


28.0 


8.6 


5.4 




6. Your preparation in your field or 
major was: 


29.1 


28.9 


51.3 


59.7 


13.2 


9.1 


1.8 






7. Your preparation for developing 
lesson plans was: 


23.5 


28.0 


46.1 


52.4 


20.1 


1^.3 


6.i 


2.4 




8. Your preparation for orgdnizing 
and developing teaching units was: 


15.5 


21.8 


50.0 


52.8 


24.7 


21.6 


5.4 


1.9 




9. Your preparation for evaluating 
pupil progress was: 


8.9 


14.2 


46.2 


48.5 


33.6 


32.5 


6.9 • 


2.8 




10. Your preparation for developing 
gand maintaining pupil intere£ . in 
class activities v.-as: 


15.0 


19.0 


52.1 

\ 


56.3 


34.3 


21.0 


.4.1 


2.2 
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11. Your preparation in the psychology 
of learning and its application to 
teaching was: 

12. Your preparation for understanding 
and handling specific behavior problems 
in your classroom was: 

13. Your understanding of the develop- 
mental characteristic of the specific 

age group which you prepared to, teach was: 

14. Your preparatirn for utilizing 
audio-visual aids and materials in your 
teaching was: 

15. Your preparation in understanding 
the factors that determine your own 
teaching personality and motivation was: 

16. Your preparation in teaching 
disadvantaged children was: 

17. Your preparation for teaching 
non-college bound students was: 

18. Your preparation for teaching college 
bound students was: 

19. Your knowledge relative to teacher 
employment procedures, benefits, and 
privileges before student teaching was: 

20. In general, the instruction which 
you received in classes in the College 
of Education was: 



1975- 

1976 



51.8 

39.. G 

51.2 

30.8 

53.1 
21.9 
33.1 
54.8 

39.8 

52.5 



1980- 
1981 



60.3 



47.0 



58.6 



35.8 



58.6 



42.7 



47.0 



55.2 



43.5 



63.8 



1975- 
1976 



27.3 22.2 



35.7 



23.2.1 



28.3 : 



23.7 



43.8 



39.5 31.9 



34.1 



18.1 



31.9 



20.0 



35.3 



18.6 



35.2 



27.5 



23.3 



37.1 



19.4 



1975- 
1976 



3.8 
12.0 

5.3 
11.0 

4.1 
21.9 
14.0 

4.4 

11.5 
7.6 



1980 
1981 




2.2 
5.8 
1.5 
• 8.4 

1.7 

7.1 
8.2 

4.1 
11.0 
3.9 
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/ -T^^^^^^^a-c^^^^^-j.^^;^^^ ^37 

EdccAtlon in b^ic scienc<^9 (physics, chealLstry, bioiogy^ and other life sciences) and ovathaiDatica. 
Rating n«sb«r (1-5). 




Wei!< in ratheciatlcs 
and all of the above 
basic sciences. 



Strong background in 
natheswitic3 through 
alg^ra and 
trigonoc^try and at 
least ons of th9 
Above basic sciences. 



Strong bac)cgroiind 
in Ciathesutics 
through calculus 
and differential 
equations and at 
leeiAt one of the 
above basic 
sciences. 



Strong bac)cground 
in nathcaatico 
through calculus 
and differential 
equations and at 
least two of the 
above basic 
sciences. 



Strong bac)cground 
in c^^thenatlcs 
through calculus 
and differential 
equations and 
basic sciences. 



Education in engineering fundainentals . Bating nudbcr (1-5) < 



Can handle only 
handbook situations. 



Able to handle 
problecxs slrdlar 
to ones with 
previous 
experience but 
hdLS difficulty 
handling new 
problecui. 



Adequate 
understixnding of 
the fundair«ntals 
/md adequate 
ability in 
relating theory 
and practice. 



Strong background 
in the theoretical 
foundation of the 
discipline and 
soT^tioes able to 
apply the 
fundasvsntals to 
new specific 
problccts. 



Strong background in 
the theoretical 
foundations of the 
discipline and 
always able to apply 
the fundamentals to 
nev specific problens^ 



Education in advanced enginoering technology of the discipline. Rating njrher (1-5). 



Docs not read current 
technical literature. 



Occasionally reads 
current technical 
literature. 



Reads current 
technical 
literature 
regiiliirly. 



?-eads current 
technical 
literature 
regularly and 
occasionally has 
a forral 
continuing 
education 
experience. 



Reads current technical 
literature regularly 
and has c forr^l 
ccntL-;ui.ng education 
experience at least: 
once a year. 



Education in professional support studies (econonucs, hu7<anities, social sciences, etc.), 
Rating number (1-S) . 



Is a one-dis«nsional 
engineer. 



Could beccri* 
interested in 
non-engineerlng 
fields which 
are relevant to 
engineering. 



Has an interest in 
non-engineering 
Xields which are 
relevant to 
engineering. 



Has an interest 
in# and )aiowledge 
off non- 
engineering 
fields which are 
relevant to 
engineering. 



Has an interest ihv 
)cnow ledge of « and 
ability to coossunicate 
with individuAls from 
non-engineering fields 
which are relevant to 
engineering. 



Education and ability for written comnunication. Rating nuisber (l*-5) . 



Is unable tc 
coccnunicate idee^ 
in writing. 



Vrrites adequately 
with difficulty. 



Writes uell wich 
ease. 



Writes well with 
ease and speedy 
and is able to 
sell ideas in 
writing. 



Has professional 
creative writing 
ability. 
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THE UNIVERSITY OF TENNESSEE COLLEGE OF NURSING 
Knoxvillc, Tennessee 



Graduates of our baccalaureate program have indicated that they are or have been 
employed by your institution. Would you please complete this questionnaire so 
that we may evaluate and improve our program. 

1. Are graduates usually able to make the transition to professional nursing practice 
in: 



i 



2 months 
4 months 
6 months 
Other 



2. Are graduates able to adapt to the ordinary routine operation of your institution 
in: 

1 raonth 

3 months 

6 months 

Other 

3. When arc graduates able to anticipate needs, orgar.ize tasks and sac priorities for 
nursing care? 

1 month 

3 months 

6 months 

Other 



4* Do graduates demonstrate the ability to assess and evaluate client needs and develop 
a plan of care accordingly? 

Yes No 

5* Are graduates abl6 to determine priorities iu nursing care? 
Yes ' No ' 

6. Do graduates readily assume the role as teacher for clients and families? 

Yes * No 

7. Do graduates recognize the need for and give appropriate priority to discharge 
planning? 

Yes No 

8* Do graduates serve as professional role models for other personnel? 
Yes No 

9. Are graduates cognizant of their role as members of health teams and establish 
collaborative relationships with others? 

Yes . No 
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ASSESSMENT OF ADVANCED EXAMINATIONS IN SPECIFIC FIELDS 
Program Area or Specific Field: 
Name of Examination: 

1. Total score 

Do you receive the mean total score obtained by your graduates on che 
exam at least annually? 

If so, what use is made of this information by your unit? 

2. Specific areas (subtests) covered in the exam (please list) : 

Do you receive the r. -an subtest scores obtained by your graduates 
at least annually? 

If so, what use is made of this information by your unit? 

If not, what steps would have to be taken to obtain the mean 5u;test scores? 
What use would be made of the subtest information if you had it? 

3. What are the reasons for having your graduates take this examination? 



4. What, if any, are the problems with making use in your unit of information 
obtained from this exam (e.g.. Is the content of the exam considered 
irrelevant to your curriculum?)? 
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LOCALLY DEVELOPED PROGRAM FIELD TEST 



If there is no externally validated examination to measure student 
achievement in your program field, please consider using broad objectives 
such as those on the attached page to develop specific student competencies 
that can be measured. 



What advantages for your unit could you identify for use of this 
approach, e.g., how could the results be used in program assessirent and 
improvement? 



What problem, if any, would be associated with developing^ administering 
and using the results of such an examination? 
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Report 



of the 



Student Outcomes Task Force on 
Student Satisfaction with Programs and Services 



I. Introduction 

The major goal of the Student Outcomes Task Force on Student Satisfaction 
with Programs and Services was to examine existing methods for measuring student 
satisfaction/dissatisfaction with programs and services and to evaluate their 
usefulness for program assessment and improvement. Achieving this goal has 
involved several more specific tasks, including: 

(1) defining satisfaction/dissatisfaction, programs and services, 
and the relationship between program quality and satisfaction, 

(2) identifying available methods for assessing satisfaction and 
outlining their strengths and weaknesses for the purposes of 
this project, and 

(3) recommending a measurement strategy for use at UTK (subject to 
review and further modification after review by department 
heads) . 

This report summarizes discussion and conclusions with respect to each of 
these specific tasks. 

II. Defining the Problem 

Defining the problem has focused on three issues: 

(1) \4hat is satisfaction? 

(2) What programs and services should be examined? 

(3) Uliat is the relationship between program quality and student 



Discussion of the nature of satisfaction has been inconclusive. The 
task force generally agreed that individuals may possess a subjective state 
of sati'^^f action/dissatisfaction and that this "feeling" may be an important 
influence on behavior. But what constitutes this sense of satisfaction and 
how it relates to behavior is unclear. Recognizing that there may be 
multiple conceptualizations of sa*:isf action, the strategy of the task force 



satisfaction? 
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was to evaluate instruments which purport to measure student satisfaction 
and derive from these measures an idea of how satisfaction has been defined. 

The range of possible programs and service with which students could 
express satisfaction is quite broad. Thus, the committee spent some time 
determining what level of programs/services was appropriate for the overall 
goals of the Kellogg Student Outcomes P.-oject. Since the major objective of 
the project is to improve program planning at the department level (or pro- 
gram level in colleges where this is appropriate), information useful to a 
department head appears to be critical. This information might include 
estimates of satisfaction with general campus or college characteristics, but 
also would have to include estimates of satisfaction with more specific 
department or program variables (e.g., availability of courses in the major, 
quality of advising in the major). 

In addition to the level of analysis, it is also important to identify 
the range of programs and services within a level which must be examined. Two 
sources were used to enumerate programs/services. First, existing satisfaction 
instruments were examined. Second, informal interviews .cLth a small sample of 
undergraduate students were conducted. 

A final issue which received attention was the linkage between program 
quality and satisfaction. Specifically, does high student satisfaction reflect 
a high quality program? If not, should one be more interested in building 
programs with high quality or high student satisfaction? mUe all task force 
members would probably emphasize the development of high quality programs, it 
is also apparent that one measure of quality is the satisfaction of students 
involved in the program. Tliis issue was not resolved, but it was agreed that 
in evaluating measures of student satisfaction attention xvould be focused on 
whether these instruments also assessed program quality in some respect. 
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III* Evaluating measures of student satisfaction 

Five existing measures of student satisfaction were chosen for review 
by the committee* These included : 

(1) Robert Pace's "College Student Experiences" 

(2) Cooperative Institutional Kes jarch Program's (CIRP) "1981 
Student Information Form" 

(3) National Center for Higher Educational Management Systems 
(NCHEMS) and the College Board's "SOIS Surveys" 

(4) American College Testing Service's "Student Opinion Survey" (SOS) 

(5) Dr. Gerald Hills and Ms. Jennifer Friday's "UTK Black 
Graduate Alumni Survey" 

Each instrument was evaluated in terms of the issues identified in the 

preceding section (i.e., how satisfaction is conceptualized, what level of 

programs the instrument addresses, and whether it measures perceived quality 

along with satisfaction). 

Pace's, College Student Experiences : This instrument is a wide ranging inquiry 
into how students spend their time in a university setting. It includes items 
measuring student background characteristics (e.g., age, major), frequency of 
various university-related activities, perceptions of the college environment, 
perceived gains in a number of areas as a result of college (e.g., gaining a 
broad general education, writing clearly) and opinions about college. The 
complete instrument includes more than 120 items which take approximately 40 
minutes to complete. 

The items contained in the questionnaire develop a broad picture of the 
individuals' college experiences and may provide a variety of useful informa- 
tion. However, few of the items are aimed at evaluating these experiences in 
a way that measures some sense of satisfaction. Only three items attempt to 
directly measure a subjective state of satisfaction with the college pxperienc 
and these items are very general (e.g., "How well do you like college?"). 




Wule infonnacion on activity patterns would be quite useful in explaining 
satisfaction, the instrument does not appear to provide any direct assessment 
of satisfaction with programs and services at either the general university 
level or the department/program level. Only if satisfaction is conceptualized 
as the frequency with which a given activity occurs could the instrument be 
used as a measure of satisfaction. (For example, students who ate at the 
student union "very often" would be classified as satisfied, while students 
who ate there only "occasionally" or "never" would be classified as dissatisfied). 

The instrument also does not provide any direct indicators of perceived 
quality of programs or services. 

CIRP, 1981 Student Information Form : This instrument measures information 
regarding background characteristics, career aspirations, and attitudes toward 
a variety of social issues. The instrument is not a measure of student satis- 
faction. The items do not deal directly with the institution being attended, 
nor do they specifically ask about satisfaction with programs and services at 
either the department or college level. Finally, there is no assessment of 
perceived program quality. Thus, this instrument was judged totally inadequate 
for the purposes of this project. 

NCHEMS, SOIS S urveys: This instrument provides questionnaires which allow 
information collection from five student groups: entering students, current 
continuing students, graduating students, former students who did not complete 
their programs of study, and recent alumni. In addition to the usual demo- 
graphic information, each form assesses student goals (academic, career 
preparation, job or career improvement, social and cultural participation, 
personal development and enrichment) and, when applicable, perceptions of 
goal achievement. 
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For all but entering students, knowledge of, use of, and satisfaction 

or dissatisfaction with the following services are assessed: 

Admissions Student Employment 

Registration Career planning 

Business office Job placement 

Academic advising Housing services 

Guidance, counseling, and testing Cafeteria 
Reading, writing, math, and study skills Health services 

Improvement Library 

Tutoring Child care 

Minority affairs Bookstore 

College cultural programs Parking 

Recreation and athletic programs Campus security 

Financial aid Other 

For each program/service students are given the following response categories: 

"I did not know about this service," "I knew about this service, but did not 

use it," "I used this service and was satisfied with it," "I used this service 

but was not satisfied with it." Thus, in this instrument satisfaction is 

measured directly. 

The questionnaire for entering students provides information regarding 
students* expectations relative to fifteen variables including cost, college's 
reputation, course offerings, availability of financial aid, and identification 
with fellow students. Information relative to the student's previous attrition 
pattern is also sought. Alumni provide information regarding outcomes of 
licensing or certification exams and perceived quality of preparation for new 
jobs or advanced college work. 

Though the instrument measures satisfaction, severa] problems exist. Only 
the former student form elicits a specific response regarding satisfaction/ 
dissatisfaction with the quality of teaching and the learning environment. No 
form explores the specific reasonb for dissatisfaction with services or teaching. 
No form allows measurement of satisfaction/dissatisfaction between colleges 
other than satisfaction/dissatisfaction with reading, writing, math, and 
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study-skills improvement. Since some colleges provide service courses for 
students from other colleges, information concerning student satisfaction 
with these prerequisite courses should prove valuable to the colleges involved 
NCHEMS-College Board provide questionnaire analysis service using standardized 
analysis packages which they describe as "available on short notice." Any 
institution planning to use that service may add fifteen additional questions 
with response values ranging from 0-9. Some of the above-mentioned defici- 
encies could be alleviated through the addition of such locally generated 
questions. Using all forms available, longitudinal evaluation could be 
accomplished by matching responses with masterfile information. 

Finally, the SOIS surveys do not include a direct assessment of 
perceived program quality. 

ACT,_Student Opinion Survey: This instrument covers three broad areas includin 
background characteristics, college services and programs, and the general 
college environment. Background characteristics include age, race, class level 
sex, marital status, reason for entering college, and related items. The 
services and programs section focuses on 23 broad program areas within che 
University such as academic advising, career planning, library facilities, 
food service, and parking. Students are asked to indicate whether they have 
used each service and how satisfied they are with it. Satisfaction is 
measured on a 5-point scale ranging from very satisfied to very dissatisfied. 
Finally, the generally college environment is tapped by questions in six topic 
areas including academic, admissions, rules and regulations, facilities, 
registration, and general. Within each topic area are several items aimed at 
different issues or areas of concern for students such as "testing/grading 
system" or "class size" in the academic area. Students indicate whether each 
area applies to them and rate the item on the same 5-point satisfaction scale 
used in the previous section. 
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The SOS questionnaire appears to present a valid methodology for assessing 
student satisfaction. The authors have been careful to design an instrument 
which takes into account both familiarity with the- topic or program and provides 
the full range of potential levels of satisfaction without being overly complex* 
However, there are some problems which may influence the validity of the results* 
First, the format of the instrument seems to encourage a response set bias 
(i*e., marking the same response category for every item). Second, the survey 
done at UT last year resulted in a response rate of only about AO percent 
(despite several follow-up efforts). This suggests that there is some problem 
in motivating students to complete the information and may also suggest that 
the information which is received does not represent a very complete picture 
of student satisfaction. Finally, from a program planning perspective, 
particularly at the department level, the items used in the survey are much too 
broad. Analysis of the responses may point to general problem areas at the 
University level, but the results provide little basis for designing policy 
changes at the University level or providing useful information to departments 
or colleges. At best, the SOS survey is a broad gauge of student satisfactions 
and dissatisfactions. For the purposes of the Kellogg Student Outcomes Project, 
more datailed information on specific programs would have to be collected for 
department and program use (which could be accomplished with the addition of 
extra questions) . 

In its present form, the SOS instrument does not provide information on 
program quality or perceived quality independent of the measure of satisfaction. 
To the extent that satisfaction is a consequence of perceiving high quality, 
data do give an indirect assessment of program quality. However, it is easy 
to imagine a student being satisfied (or dissatisfied) with a program without 
having given much thought to assessing the quality of the program. 
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Hills and Friday, DTK Rl.ack Graduate Alumni ^nrv.v This instrument is 
designed to gather Information on Black students who have received post 
graduate degrees from UTK. Items obtain information on background charac- 
teristics, current job or school status, and satisfaction with a large variety 
of programs, services, and characteristics of the University of Tennessee. 
This instrument has several innovative features which appear useful for the 
Student Outcomes project. 

First, the instrument attempts to measure both satisfaction with a 
specific program or service and the importance of that program or service 
in an overall evaluation of the University. Second, the instrument measures 
expected level of satisfaction with specific programs before entering UT 
(through a retrospective question), and actual satisfaction with the program 
after leaving. Third, the instrument attempts to measure program quality at 
several points. 

Although the instrument would require revisions for use with graduate 
students, and probably would require some type of validation studies, it does 
provide a useful alternative to more simplistic estimates of program satis- 
faction. The major drawbacks of the instrument are that it is complex (and 
thus may be hard to understand for many undergraduates) and it does not assess 
why individuals are satisfied or dissatisfied. 

Summary of Instrument Evaluation. , with regard to the objectives of the 
Kellogg Student Outcomes project, the review by task force members suggests 
a number of important points. First, instruments differ greatly in the types 
of information gathered, but three do provide fairly direct assessments of 
student satisfaction with programs/services. (They are SOIS, SOS, and UTK 
Black Graduate Alumni Survey). Second, no_ne of the instruments provides very 
detailed measures of satisfaction at the department or program level, though 
the three which measure satisfaction directly provide useful information 
regarding satisfaction at the University level. Thus, as discussed in the next 

ins 



100 



section, questions will have to be designed to gather information useful to 
department and program heads. Third, only the UTK Black Graduate Alumni 
Survey attempts to assess perceived program quality in addition to satisfaction/ 
dissatisfaction* If the ultimate goal of the Student Outcomes program is to 
gather information useful to improving program quality as well as increasing 
student satisfaction, it will be important to design an instrument which taps 
both concerns* 

It should also be noted that the review leaves us no closer to an under- 
standing of what constitutes satisfaction, only a way to determine whether 
respondents feel satisfied or dissatisfied with a particular program. None of 
the instruments systematically measures why individuals feel satisfied or 
dissatisfied. Developing a framework for explaining why students are satisfied/ 
dissatisfied must be preceded by a clear definition of what satisfaction is. 
Thus, defining satisfaction should be an important objective in refining the 
measuring instrument. 

IV. Recommendations for Measuring Student Satisfaction at UTK 

This section summarizes the recommendations of the committee regarding 
the measurement of student satisfaction. Two issues are addressed Hn this 
discussion: instrument selection and design, and methodological design. 
Instrument Selection and Design . Comparison of the relative strengths and 
weaknesses of five existing measures of student satisfaction led task force 
members to conclude that the ACT "Student Opinion Survey" (SOS), which 
already has been used twice (Spring 1980 and Spring 1981) at UTK, is more 
likely than any other currently available standardized instrument to provide 
the kind of general information about student satisfaction that can be used 
in program assessment and improvement at UTK, The SOS (see Appendix A) 
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utilizes an attractive, readable format; it measures student satisfaction 
directly; it provides a measure of the extent of usage, as well as level of 
satisfaction, for 23 broad program/service areas; and finally, it includes 
a set of more specific questions designed to assess satisfaction with each 
of six aspects of the University environment. It costs approximately $7 
per student sampled for ACT to administer and score the SOS and to provide 
a technical report of findings. 

While the ACT SOS can provide a measure of student satisfaction on a 
University-wide basis, the information it yields is not sufficiently specific 
for use by deans or department heads in assessing their own programs and 
services. Accordingly, task force members developed, in consultation with a 
sample of department heads, a set of questions that could provide information 
specific enough for unit level evaluation. Ihese items could be used in a 
form suitable for enrolled students and a form for recent graduates. A 
preliminary set of questions, with a sample response format, appears in 
Appendix B as the "Unit Level Survey of Student Satisfaction with Programs and 
Services." As indicated below, further development of this instrument, including 
an extensive pilot test will be required in order to evaluate its content and 
format. 

Finally, since the task force views the evaluation of teaching as an 
important component of the information concerning student satisfaction to which 
deans and department heads must have access in order to assess the totality of 
the academic experience for students, the members recommend that instruments 
for the evaluation of teaching that have been developed by departments or 
obtained from the Learning Research Center be used within each unit. 

The ACT SOS, the "Unit Level Survey," and instruments for the evaluation 
of teaching have similar purposes and thus contain many similar items. If 
any combination of these instruments is to be administered to a given sample of 
students, forms of each should be altered to minimize duplication of content. 
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Methodological Design . The ACT SOS should be administered at least every 
other year to representative samples of (1) enrolled students and (2) recent 
graduates of UTK. The .'asulting information should be reviex^7ed by an appro- 
priate body such as the University-wide Instructional Evaluation Committee. 
Following comparison of current data with that obtained in previous adminis- 
trations of the SOS, and perhaps some follow-up inteirviewing to increase 
understanding of certain responses, the oversight grojp should issue a report 
on the findings to deans and department heads, including recommendations for 
program improvements which seem warranted. 

In order to field test the "Unit Level Sarvey" and determine appropriate 
sampling methodologies for programs in the variety of disciplines represented 
at UTK, a pilot testing phase is recommended. 

Approximately 10 units interested in participation should be selected to 
field test the "Unit Level Survey" in classes within the unit. Each college 
or department must, of course, determine its own method of sampling students, 
but the follox^ing methodological considerations are suggested. 

A. Because students* motivation may be related to their purposes 

in taking courses it may be useful for the unit to differentiate 

the responses of several types of students, i.'^., 

. majors , 

. non-majors who are taking a course within the program of the 
given unit because that course is required in their own program, and 
. non-ma j ors taking a course as an elective . 

B. Keeping in mind the different purposes for which students are 
taking courses, a representative sample of courses in which the 
"Unit Level Survey" will be administered should include service 
courses as well as those likely to include a high percentage of 
majors. A sample of approximately 300 students, or all students 
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if less than 300, should be drawn ♦ Students at the four classi- 



fication levels — freshman, sophomore, j 



junior, and senior — 



should be sampled as nearly as possible in proportion to their 
numbers in the population of students enrolled in the unit's 
courses ♦ 

C* The "Unit Level Survey'' could be given at the same time, and to 
the same students, as is the evaluation-of-teaching instrument 
selected for use by each instructor. 
A group of faculty with interests in curriculum development, program 
evaluation, measurement, and survey design and analysis should be established 
to oversee the pilot test of the "Unit Level Survey/' This group should 
approve a response format for the instrument, select the 10 pilot test units 
and make suggestions for improving proposed course sampling methodologies in 
the various units, and develop procedures for analyzing the data and reporting 
it to unit administrators and faculty. 

The "Unit Level Survey" should be given to a representative sample of 
enrolled students at least every other year, and to a sample of recent gradu- 
ates (out of school 1 to 3 years) of each unit at least once every four 
years. The information thus obtained could be used by deans and department 
heads to suggest the following: 

a) changes in curriculum at the course level or at the program ?3vel; 

b) areas for professional development of faculty; 

c) adjustments in testing/grading procedures; 

d) adjustments in class sizes; 

e) the addition of, or changes in, field experiences such as practica, 
internships and apprenticeships; 

f) changes in advising procedures; 

g) improvements in classroom or laboratory facilities and/or equipment. 
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USING THE IIST OF COLI EGE 
MAJORS AND OCCUPATIONAL 
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THIS OUEStlONNAinE. TLFASE 
StlCCT THE THREE DIOIT 
COOES FOR YOUR COLLEGE 
MAJOR AND YOUM OCCUPA. 
TIONAL CHOICE. WRHE THESE 
CODES IN THE BOXES AT THE 
TOP OF niOCKS O AND P. 
AND ULAC^EN THE Al'PHO. 
PRiAtE OVAL IN THE COLUMN 
lULOW EACH OOX. (IF YOU 
HAVE MORE THAN CrJE MAJOR. 
SELECT THE ONE CODE THAT 
8EST DESCRIOES YOUR 
EDUCATIONAL PROGRAM ) 



0 



INOICAIE 
YOUR COLLEGE 
MAJOR 



III 



INDICATE YOUR 
OrCUPAllONAL 
CHOICE 
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PAnr A: USAGE 



COMPLETE PART 6 ONLY IF YOU 
H/^VE USCO THE SERVICE 
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UNIT LEVEL SURVEY 
OF 

STUDENT SATISFACTION WITH PROGRAMS AND SERVICES 



Course Being Evaluated 

Your classification: Freshman Sophomore Junior Senior Graduate 
Are you a major in this department (or college)? Yes No 

Is this course required for your program of study? Yes No 

Are you taking this course as an elective? Yes No 




PROGRAMS /SERVICES 



How satisfied are you with 
this program/service? 



Very 
Dis, 



Dis. 



Neutral 
or 

No Opin. Sat. 



RAIINGS 



Quality of Instruction 

U Course content (overall) 

a. Comprehensiveness of course content 

b. Relevance of content for my needs 

c. Extent to which content is current, up-to-date 

d. Extent to wr»ich content is repetitive of work 
I've done in other courses in this unit 

2. Quality of teaching in this course 

3. Appropriateness of the testing/grading system in this course 
4* Appropriateness of the size (number of students) of this class 

5. Value of this course in preparing me for employment or for 
further study in this academic area 

6. Availability of the course instructor (s) to help me outside class time 

?• Willingness of the course instructor (s) to help me understand course conr^at 
8. Fairness (impartiality) of the instructor (s) 
Instructor interest in students as individuals 



Very 
Sat . 



How important is this program/ 
service in your overall evaluation 
of your experience at UTK? 



Very Very 
Unimp . Unimp. Neither Imp> Imp. 
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Neutral 

Very or 
Dls. Dls. No Opin. 



Very 
Sat. 



Quality o f Facilities or Serv ices Dls. Pis. No Opin. Sat. 

i. Adequacy of classroom facilities for this course 
'2. Adequacy of laboratory/studio/clinical facilities for this course (if applicable) 

3. Adequacy of equipment available for use in laboratory/studio/clinic (if applicable) 

4. Adequacy of audio-visual aids used in this course (if applicable) 

5. Adequacy of the library collection related to this course 

^' ?of ^Sis1ourse''°'' providing text and supplementary materials 



Very 

Unimp 



Unlmp> Neither I^^^C^ 



O 
00 



If you are a major in the department/college in which this course is offered, please respond to the 
following additional items: 



Quality of Instruction 

1. Adequacy of preparation provided by lower division (1000 and 2000 level) 
courses for upper division (3000 and 4000 level) courses in your major 

2. Lack of repetition among courses in this unit 

3. Consistency of testing/grading standards across courses in the unit 

4. Consisnency of testing/grading standards in this unit as compared with standards 
m other units across the University omparea wicn standards 

5. Quality of courses in this unit for providing a well-rounded education 

6. Quality of courses in this unit for preparing me for employment or for further study 

7. Extent of my involvement in faculty research or other scholarly endeavor 

8. Availability of the courses I need to take in this unit 

9. Logic of course sequencing and curriculum organization 

10. Variety of courses offered in my major 

11. Availability of the courses in my major 

12. Availability of optional courses in my program 
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13. Are seminars or colloquia offered for undergraduates in, this area? Yes No 

If not, would you like to have seminars or colloquia? Yes No 

14. Is a practicum, internship, or field experience provided in your major? Yes No 

If so, how satisfied are you with the experience? Very Dis, Dis, Neutral Sat. Very Sat. 

If not, would you like to engage in a field experience as part of your program? Yes No 

15. Is a comprehensive or licensing exam required of you during or following your senior year in your major? Yes No_ 

If so, how satisfied are you with the preparation for this exam which is provided by this unit? Very Dis. Dis . 

Neutral Sat. Very Sat. 




Quality of Faculty/Staff Assistance 



Neutral 

Very or Very Very 

Dis. Dis . No Opin. Sat. Sat. Unimp. 



Very 

Unimp. Neutral Imp. Imp. 



my 



1. Quality of academic advising provided by this unit (overall) 

a. Availability of my advisor 

b. Willingness of my advisor to provide the help I need 

c. Quality of assistance my advisor has provided concerning 
degree or program plan 

d. Quantity of assistance my advisor has provided 

e. Consistency of the information provided by different advisors in this unit 

2. Quality of printed information concerning programs offered by this unit 

3. Helpfulness of unit staff in providing the information or assistance I need 

4. Adequacy with which complaints are handled by faculty/staff in this unit 

5. Quality of special events sponsored by this unit (e.g., guest speakers, exhibits, field trips, etc.) 
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APPENDIX 6 



Pilot Project 
Request for Proposals 
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[Q OFFICE OF THE CHANCELLOR 



October 7, 1982 



Dean William Coffield 

212 Claxton Education Building 

Campus - 3400 

Dear Bill: 

On August 23 you and Tom and I discussed several ideas that might be 
developed into pilot projects with support from Kellogg funds. Specifically, 
we covered the following possibilities: 

1. Giving the advanced National Teacher Exam in elementary education 
to a sample of seniors majoring in elementary education. 

2. Selecting or developing a comprehensive exam in recreation, dance> 
health and safety, distributive education, industrial training, or 
one of the other fields for which the NTE is not appropriate. 

3. Conducting a survey of enrolled students and/or alumni in a given 
department. 

4. Conducting exit interviews with graduating seniors. 

In the weeks since we talked other ideas fo.r acquiring and/or using 
student outcomes infomation may have occurred to you. 

I am enclosing copies of proposal guidelines and an application forra 
which will assist us in the process of awarding grants for pilot projects 
in the second phase of the Kellogg Student Outcomes Project. Please use this 
form in developing the project proposal you x^ould like to pursue. 

Given available resources and needs I can foresee in connection with any 
one of these project ideas, I would suggest that you consider a request for 
approximately $500 of Kellogg funds. In actuality the need for funds may 
turn out to be slightly more or less than that. Since the application review 
process will focus on alloca _on of available resources rather than selection 
among competing proposals, please develop the best project that can be con- 
ceived, along with the needed budget, then if we find that there is a need 
to reconsider the amount allocated, we will do that. I will be glad to 
discuss with you any details of the application or the review process. 
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Dean William Coffield 
October 7, 1982 
Page 2 



Please send the application to me by November 5, A committee will review 
the proposal and communicate with the contact person by early December con- 
cerning the request for resources. 

Projects may begin as early as January 1, but should be completed no 
later than September 1, 1983. I look forward to receiving your proposal. 

Sincerely, 

Trudy W. Banta 
Professor ^ 

Special Assistant to the Chancellor 

It 

Enclosures 

cc: Dr. Thomas George 



I 
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GUIDELINES FOR PROPOSALS DESIGNED TO INCREASE AND IMPROVE 

USE OF INFORMATION DERIVED FROM MEASURES 
OF STUDENT ACHIEVEMEWT IN THE MAJOR FIELD AND/OR STUDENT 
ASSESSMENT OF THE QUALITY OF PROGRAMS /SERVICES 



Through the Kellogg Student Outcomes Project small grants of $350 to $1,500 
each will be made to colleges and departments submitting proposals for projects 
designed to increase and improve the use of stuodnt outcomes information of one 
or both of the following types: 

(1) student achievement in the major field 

(2) student assessment of the quality of p-^-ograms and services 



Student Achievement in the Major Field 



Student achievement in the major field may be measured using a standardized 
examination such as the National Teacher Exam, a licensing exam, or a GRE 
advanced test. Though probably more costly, difficult, and time-consuming, 
a unit may elect to design its own comprehensi/e field exam. In addition, one 
or more of the following methods may be employed: 

1) Evaluation by faculty of a comprehensive student achievement 

2) Evaluation of a comprehensive student achievement by an external reviewer 

3) End-of-program assessment by seniors, reporting perceptions of their own 
achievement 



4) Retrospective assessment of their achievement by alumni 

5) Assessment by employers of the competencias of alumni one or several 
years after graduation 

A unit proposing a pilot project in this outcome area might request one or 
more of the following types of assistance from the grant program: 

1) guidance in identifying and/or selecting an appropriate field exam 

2) funds to purchase a standardized exam fcr administration to a sample of 
advanced students 

3) guidance in developing a measurement instrument 

4) assistance in obtaining from licensing or accrediting agencies additional 
information about student scores which would enhance the usefulness of 
these data in program assessment 

5) guidance and/or funding for additional analyses of existing student 
achievement data 

6) guidance in making additional applications to program assessment and 
improvement of student achievement data collected previously 
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Student Assessments of the Quality of Programs/Servi 



ces 



in tnis outcome area a unit may propose to construct or select for use 
a new instrument, improve the design and/or administration of an instrument 
used previously, or improve the application to progfam assessment and improve- 
ment of information derived from an instrument used previously. One or more 
0^ the following types of measures may be involved: 

1) assessment by students (majors, non-majors taking required courses, 
non-majors taking electives, in the unit) of the quality of programs 
and services at the departmental or college level (for example, 
instruction, advising, internships, course sequencing). 

2) assessment by alumni of the quality of programs/services at the 
departmental or college level. 

In this connection a unit may request one or more of the following types 
ol assistance: ^ 

1) guida.-ce in selecting or designing a questionnaire or employing an 
appropriate sampling methodology 

2) funds to duplicate a questionnaire, mail it to a sample, and follow 
up to ensure an adequate return 

3) guidance in analyzing, interpreting, and reporting the results of 
student survey data and in applying the findings in program assess- 
ment and improvement 



The Proposal 



As stated previously, a proposal may suggest methods of increasing and 
improving the use of information derived from (1) measures of student achieve- 
men^ in the major field, OR (2) student assessments of the quality of programs/ 
services, OR a combination of 1 and 2. h ^ v i 

f^?!" ^ addressed in the proposal, brief responses to 

the following items should be included: 

1) ^^mat kinds of student data are already being collected by the unit? 

2) How arc these data currently being applied in program assessment and 
decision-making? 

3) What changes (s) or improvement (s) in data gathering methodology or in 
application strategies are proposed? Specify objectives to be accom- 
plished . 

4) How will the proposed changes increase and improve the _use made of this 
kind of information in combination with other kinds of information in 
program assessment and decision-making? 
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Student outcomes information can be applied or used to provide evidence of 
program quality for the following purposes: (a) communicating with alumni and 
friends of the University, students and their parents, professional colleagues; 
(b) preparing for accreditation reviews; (c) strategic planning; (d) preparing 
for the comprehensive program review process; (e) responding to the requirements 
of the THEC Instructional Evaluation Schedule, 

More specifically, outcomes information can be used by a unit to suggest: 

1) Changes in curriculum at the course level, i.e., course content updated 
or given new focus 

2) Changes in curriculum at the program level, i.e., courses or content 
areas added or deleted 

3) Areas for professional development of faculty 

4) Addition of, or changes in, field experiences 

5) Addition of, or changes in, seminars, colloquia, special events 

6) Changes in advising procedures 

7) Improvements in classroom or laboratory facilities and/or equipment 

8) Additions to library collections 

9) Improvements in campus-wide student services 

Finally, the proposal should contain an estimate of the quantity (time 
and/or dollar value) of the following types of resources that are needed to 
accomplish the project objectives: 

1) Technical assistance (Faculty/administrator time and/or graduate assistant 
time — specify whether this expertise can be obtained in the unit or 
must be obtained externally 

2) Financial assistance for instrument pui\.hoSe and/or administration 
(Include costs of postage, telephone follow-up, or monitoring of exams.) 

3) Data processing assistance (Specify type(s) and quantities of data to 
be processed.) 
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APPLICATION 



Kellogg Student Outcomes Project 

Name of Unit | 

Contact Person Campus Address Telephone 

Please supply brief responses (one or two paragraphs) to the following items. Applida 
may cover area 1 or area 2 below, or both. 

i 



1. Student Achievement in the Major Fie** d 

a. What kinds of infor 
the unit already have? 



What kinds of information on student achievement in the major field does ;l 
^ unit already have? W 



How is this information currently being applied by the unit to assess and J 
improve the quality of its programs? §. 



What change(s) in information-gathering or application strategies (a or b 
above) do you propose? Briefly outline the strategy including objectives I 
to be accomplished; instruments you plan to use; characteristics of the ^ 
student sample; and methods of data collection, analysis, and interpretation! 
to be used. 



Student Assessments of the Quality of Programs/Services 



u 



a. 



What kinds of information does the unit already have that could be classifiejl 
as student assessment of the quality of programs/services? || 



b. How is this information currently being applied by the unit to assess and 
improve the quality of its programs/services? 



IThat change (s) in information-gathering or application strategies (a or b 
above) do you propose? Briefly outline the strategy including instruments; . 
student sample; and methods of data collection, analysis, and interpretation 1 
to be made. 4 



How will the change(s) proposed in Ic and/or 2c above increase and improve the 
tiSS made of these kinds of information in combination with other kinds of 
information in the assessment and improvement of program quality? In this 
response please outline a 3-5 year plan for using student outcomes information 
of the type specified if funding were available to continue the methodology 
you have proposed. 
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4* Proposed starting date for project. Ending date 



5* Type(s) of Assistance Requested - Describe briefly the resouces to be used 
in each applicable category and estimate (1) the amount of Kellogg funds 
needed to provide those resources, and (2) the cost of the in-kind College 
contribution that is anticipated (principally time of faculty and adminis- 
trators that will be involved. 



Category of Assistance Kellogg Funds College Funds Total 

a. Technical assistance (Estimate the 
cost of time for faculty, adminis- 
trators, graduate assistants and/ 
or outside consultants to provide 
assistance with instrument selec- 
tion or design; survey work, data 
analysis, interpretation, etc.) 

b. Supplies, postage, telephone 



c. Travel 



d. Computer time 



e. Other 



Please return completed Application by November 5, 1982 to: 

Dr. Trudy W. Banta 
541 Andy Holt Tower 
Campus - 0150 
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[D OFFICE OF THE CHANCELLOR 



THE UNlVEPSlTv CF TENh JE3SEE, kNOX\/ILI£/ 527 ANDY HOLT TCWEr^/'K'^:OXMLLE. TENNESSEE 37996-C oO 



MEMORANDUM 



TO: 



Dr. John Prados 



FROM: 



Jack E. Reese W 
Chancellor (\ 

January 24, 1983 




DATE: 



RE: 



Testing Requirement for UTK Baccalaureate 



The UTK Faculty Senate, following the recommendation of our Instructional 
Evaluation Committee through the Undergraduate Council, has voted to 
require, beginning with Spring 1983, that every candidate for the 
baccalaureate must participate in the UTK instructional evaluation 
program, by taking either the ACT-COMP test, or a test of competence 
in a specific major field of study, or by completing some other instrument 
designed to determine the perceptions and reactions of seniors to the 
course of study they have pursued or to their experience at the University. 

These are activities required not only by the Instructional Evaluation 
Schedule of the Tennessee Higher Education Commission, but by our own 
commitment to the improvement of teaching and learning on this campus. 
We believe that graduating students, who have enjoyed a considerable 
subsidy from the people of the state of Tennessee in covering their 
educational expenses, will in the large be willing and in most cases 
eager to be a part of this extensive effort to Iniprove the quality of 
the baccalaureate degree. We value their insights and perceptions now, 
as we will value their perceptions when they become loyal alumni over 
the years. 

We will appreciate your bringing this request to the attention of the 
Board in February. I recommend that the Board consider the following 
Catalogue statement of the requirement: 



In order for the University to assess and improve its academic 
programs, periodic measurements of student perceptions and 
intellectual growth must be obtained. As a requirement for 
graduation every student shall participate in one or more 
evaluative procedures, which may include examinations in 
general education and/or the major field of study. The 
evaluative information obtained through testing is one of 
the means used to improve the quality of the educational 
experience for future generations of students. 
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APPENDIX 8 



Report 



on 



Student Satisfaction Survey 
June 1983 



Kent Van Li ere 
William Lyons 
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^22 Report 

on 

Student Satisfaction Survey 
June 1983 

OVERVIEW Van Liere 

William Lyons 

The purpose of this report is to summarize findings of a study 
of student satisfaction with University programs and services. The 
study was conducted in the Spring quarter, 1983 and is based on a 
sample of full and part-time undergraduates at the University of 
Tennessee-Knoxville . Before summarizing the findings, a brief 
overview of the questionnaire and methodology are provided^ 

The questionnaire was designed to measure satisfaction with 
programs and services in three different areas. First, items were 
included in the survey which nieasure satisfaction with general 
University services and programs such as the library, health service 
and campus plays. with '/espect to each service, students were 
asked to indicate their frequency of use, reasons for nonuse, 
ratings of the quality of the service, and the importance of the 
service. Second, students evaluated services and programs in their 
major or intended major such as advising, availability of courses, 
and faculty-student interactions. Students unable to specify a 
major or intended major simply skipped this section. Third, 
students rated the quality and importance of various facets of their 
classroom experience. Students were asked to rate a specific class 
randomly chosen from the classes they were taking during Spring 
quarter . 

In addition to tapping satisfaction at these different levels, 
the questionnaire also asked about overall satisfaction with the 
academic and social environment at UTK. Finally, a series of 
socio-demographic questions were included. This report summarizes 
the responses to these different groups of items. (A copy of the 
feRXC^^""^^^^ is included in the Appendix) . 



The data were collected between April 25 and June 8, 1983. The 
responses in this study are based on a random sample of 1155 
undergraduate students (full and part-time, enrolled in classes 
during Spring quarter. The sample was stratified by college, since 
several colleges represent fairly small proportions of the total 
student population at UTK, a disproportionate sampling procedure was 
used to oversample small colleges. The following sample sizes were 
used in constructing the sample: 

College of Agriculture n=100 

College of Architecture n=100 

College of Business n=150 

College of Communication n=100 

College of Education n=100 

College of Engineering n=l25 

College of Home Economics n=100 

College of Liberal Arts n=250 

College of Nursing n=lO0 

Unclassified n=30 
in reporting the results for the Univeristy as a whole, the 
college samples were weighted to reflect their true proportions 
for the University. 

The questionnaire was distributed through the mail. An 
initial mailing and two follow-ups were used to solicit 
responses. The first mailing included the questionnaire, a 
cover letter on Chancellor's office stationary, and a 
postage-paid return envelope. The second mailing was a postcard 
reminder. The third mailing included another copy of the 
quesitionnaire, a cover letter, and a return envelope, of the 
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1155 students in the sample^ 809 returned useable questionnaires 
for a response rate of 70*0 percent. 

In addition to the general sample of University students^ 
the study also included additional samples of majors and 
students in classes for five departments or programs which 
received a more indepth evaluation. Results from these five 
departmental studies are summarized in separate reports prepared 
for each department. 
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SUMMARY OF FINDINGS 

The purpose of this section is to draw attention to major 
findings which appear relevant to the University* s response to 
the THEC guidelines. The Appendix provides detailed tables 
summarizing the date on which conclusions are based. It is 
important to note that much of the specific information included 
in the tables will be of most interest to those managing the 
various programs and services evaluated. 

Evaluation of General University Programs and Services; In 
discussing these services we will first discuss overall 
evaluations/ then examine frequency of use and reasons for 
nonuse. Throughout this report ratings of quality are based on a 
four-point scale from excellent to poor. Services receiving the 
highest percentage of "excellent" or "good" ratings from 
students using the service included the main and undergraduate 
libraries^ campus plays/ films and concert series^ and the 
University bookstore. For all of these services^ over 70 
percent provided excellent or good ratings. 

Services which received a high proportion of "fa^r" or 



"poor" ratings included career planning and placement, 
counseling center, writing lab, student employment service, 
health service- and computer services • For these services, over 
40 percent of the respondents gave fair or poor evaluations of 
the quality of the service* Relative to the other services 
evaluated, the health service seems to have the most problems, 
as 17 percent of those using the service gave poor ratings* 
However it is important to note that 20 percent of the sample 
gave the health service an excellent rating, suggesting that the 
quality of service is, at worst, uneven. 

In general, the evaluations reported above are relatively 
consistent across colleges, classification of students 
(freshman, etc.), and type of student based on low versus high 

In interpreting the evaluation of various services, it is 
important to consider the extent of use by studen*-.s and the 
reasons for nonuse. Among the services which received more 
positive evaluations, the libraries and the University bookstore 
were most frequently used* All respondents reported using the 
bookstore to some degree, only 1 percent reported never using 
the undergraduate library, and 17 percent never use the main 
library. Two-thirds of the sample indicated that they took 
advantage of the campus plays, films, and concerts on at least a 
few occasions. Among the services receiving weaker ratings, the 
counseling center, the writing lab, the career planning and 
placement center, and student employment services were used 
infrequently by students. Three- four ths of the sample or more 
had never used these services. 
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Why weren't these services used more? For most services, 
students siir.ply had no further need for the service. However, 
for three services-career planning and placement center, 
counseling center, and the writing lab-over 50 percent said 
they were not familiar with the service. Apparently there is a 
need for more information regarding these services among 
students. Also, 16 percent of the sample reported not using the 
health service more frequently because they had found the 
services to be of low quality. These results combined with 
results reported above suggest a need to improve the health 
service, or at least student impressions of this service. 

A second set of general University services which do not 
involve frequent interaction with students such as admissions, 
student records, drop/ add, and student scholarships was also 
evaluated. For these services student were asked to evaluate 
the quality of the service and to rate the clarity of 
procedures. Services receiving more than 70 percent excellent 
or good ratings were admissions, student records, and 
preregis .ration, students also rated the procedures for these 
three services as clear or only somewhat confusing. Very few 
students, less than 10 percent, rated these services as poor, 
in contrast, several services received poor ratings by 20 
percent or more of the sample. These included registration,, 
student loans, grants, and scholarships, the student conduct 
office, and drop/add procedures. Approximately one-tnird of the 
sample rated the procedures for these services as confusing. 
These ratings are based only on the responses of students who 
were familiar with the service. 

ErJc . 1^8 



Ratings of Prcgra.s and Services in the Major: The survey 
also included a set o£ items designed to measure student 
satisfaction with the programs and services in their major. 
Although this type of information will be most useful to 
department or program heads, a University-wide su-ary of the 
data are included in this report because it provides a general 
description of the state of these services in the University and 
can be used as a baseline for comparing department responses. 

Students were as.ed to rate the quality of each service as 
well as indicating its importance to them, students who could 
identify either a major or an intended major completed this 
section. Approximately 12 percent of the students were 
undecided about their major and skipped this section. 

seventeen services were included in the questionnaire, 
services receiving the highest average ratings of quality 
included the following: 

quality of instruction in upper division courses (average 
rating of 3.1) 

willingness of the advisor to help (2.9) 

quality of course for providing a general education (2.9) 
Items with the lowest average ratings included: 

quality of printed program information (2.6) 
quality of special events (2.6) 

quality of instruction in lower division courses in the 
major (2.6) 

opportunities for interaction with faculty in the major 
(2.6) 

1:^3 
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practicum end internship experiences in the major (2.6) 
availability of required course in the major (2.4) 
availability of desired courses in the major {2.4) 
In terms of importance to the student, the three most 
important services were: quality of courses in preparing the 
student for employment, quality of instruction in upper division 
courses in the major, and the availability of required courses 
for the major. Given the importance of this last item to 
students and its low quality rating, this area deserves more 
attention in reducing student dissatisfaction. 

Interestingly, the quality ratings of many of these 
services do vary by college (see Table 12 in the Appendix). 
Thus it may be appropriate to work with the administrators in 
the various colleges to design programmatic and administrative 
changes to respond to weaknesses within colleges, rather than to 
initiate University-wide changes. Surprisingly, there was 
little difference in the ratings of the services provided in the 
major by students of different classification { i .e ., freshman , 
etc.) or by students of different grade point averages. 

Ratings of the Quality of Facets of Classroom Experience: 
The survey also contained a set of items evaluating the 
classroom experience. This set of items was very similar to a 
standard class evaluation form. Rather than have students 
respond to these items generally, a specific class was randomly 
chosen from the list of classes a student was enrolled in Spring 
quarter and students were asked to evaluate this class (the name 
of the Class was written on the questionnaire). Thus responses 
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to this set of questions should provide a fairly representive 
picture of student evaluation of classes .across the university 
as a whole, students were asked to evaluate both the quality of 
the service and its importance to them. 

Items which received the highest average rating included 
the following: 

instructor's knowledge of the subject matter (average 
rating of 3.6) 

instructor's enthusiasm for teaching this class (3.3) 
instructor's class preparation (3.2) 
extent to which content is current (3.2) 
overall quality of the instructor (3.2) 
Items which received the lowest overall ratings included: 
relevance, of content for student needs (2.8) 
clarity of course objectives (2.8) 
conduciveness of climate for learning (2.8) 
quality of classroom discussion (2.7) 

accuracy of the catalog description of this course (2.7) 
The most important facets of the classroom experience 
according to the student ratings of importance included: 
instructor's knowledge of the subject, instructor's helpfulness 
with problems, instructor's class preparation and presentations, 
and the fairness of testing and grading. 

in contrast to the ratings of services in the major, there 
were few sizeable differences in the ratings of classroom 
experiences across colleges. Also there were few differences 
across students of different classifications or grade point 
averages. 
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we also asked students to indicate whether the course they 
were evaluating was taught by a graduate student or faculty 
.e.ber (they could also respond with not' sure,. A comparison of 
the differences in evaluations for faculty and graduate students 
Shows that on .ost items faculty do have higher ratings than 
graduate students, but the differences are generally not very . 
large for most items. 

However, the largest differences occur on two important items. 
Faculty receive higher ratings on both enthusiasm for teaching 
the course and knowledge of the material. 

Ratings of Overall Satisfaction with the University: The 
final set of items on student satisfaction involved general 
feelings of satisfaction with the University. The following 
suinmarizes responses to these questions; 

in general how satisfied are you with your academic 
experience at the University of Tennessee? 
very satisfied 19.5^ 
somewhat satisfied 52,2% 
somewhat dissatisfied 23.0% 
very dissatisfied 5^3^ 



jERIC 



j42 



In general how satisfied a-o 

at.sfied a.e you with your overall 

experience at the University of Tennessee, 

very satisfied ,3^,^ 

somewhat satisfied 57^7^ 

somewhat dissatisfied 12.5% 
very dissatisfied 

The important point in these results is that th 

good deal less -..^ . "^^""^ t° a 
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|, PROGRAM/SERVICE 



I Library 

I Und»rgradu8t« 
g Library 

Cara«p Planning & 
.Placamtnt Cantar 

#'ODunsaling 
|Ctnrar 

sRacraation or 
Intramural 
|Aet|y|ty 

|Hrlring Lab 

Studant 

;EMployaant 

|S«^lea$ 

|y 

Itoalth Sarvlcas 

k 

iMpus Plays 
S»»pu$ Film Sarlas 



pus Concart 
ifrlas 



putar Sarvicas 



nivarslty 
tbkttora 



TA8L£ I 

STUOEHT USE OF GENERAL UNIVERSITY PROGRAMS 
ALL STUDENTS 



AND SERVICES: 




Sarvica was ns«ri. 



FREQUENTLY 



Hi 

48J 

3S 

2f 

26S 

If 

31 

7f 
9f 

121 

lOJ 
13? 



OCCASIONALLY 



38J 
40f 
6S 
4S 

27J 
2f 

91 

25J 
25J 
32J 

24 J 
I4J 



SELDOM 



30J 
HI 
I5f 
I5f 

20f 
4f 

»4f 

28J 
30J 
26J 

27J 



NEVER 



I7J 
IS 

77J 

7« 

27J 

95% 

74J 

39J 
37J 
30J 

39J 
59J 
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P«OGRAM/SE«v,CE UNIVERSITY ""f * Col. Col. Col. 



Library* 

Unatrgroduat* 
Library* 

Carwir Planning 4 
Placereant Cant«r« 

Counsel ing 
C«ntar« 

R«cr«atlon or 

•ntramopoi 
Activity* 

writing Lsb 

Studant 

Eitploywent 

S«rvlc«s 

H««lth Services 

Campus Plays" 

Campus Film Sarlw* 

Campus Concert 
Series 

Computer Services* 

l*»lverslty 
Bookstore 



Col. Col. Col. 

G H T 



2.5 
3.4 
1.4 

>.3 

2.5 

M 

1.4 

2.0 
2.1 
2.3 



2.5 2.5 2.3 



3.5 3.5 
'.3 1.4 

>.3 1.5 

2.4 2.7 



3.2 
1.4 
1.3 
2.9 



1.1 



•.-« 1.3 



>.5 



1.5 1.4 
>.» 1.2 

2.7 2.2 

>.» 1.1 

'.5 1.5 



2.0 


2.0 


2.2 


2.0 


2.2 


2.2 


1.9 


1.9 


>.8 


2.2 


2.4 


2.1 


2.2 


2.4 


2.1 



3.0 


2.8 


2.3 


9 /I 


3.0 


3.0 


3.6 


J. 4 


1.2 


1. 1 


1.5 


1.2 


t.2 


1.3 


M 


t.2 


2.6 


2.4 


2.4 


2.4 


M 


I.l 


t.2 


t.t 


>.4 


1.4 


1.3 


t.4 


2.0 


2.4 


2.0 


2.1 


2.0 


2.0 


2.0 


2.5 


2.5 


2.1 


1.8 


2,3 




. ^'S"- --s indicate greater use 



1^ 



'11 



11 



^11 
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TABLE 3 



STUDENT USE^F gNERAL UN,,^^ ^^^^ 

MtANS BY CUSS AND GRADE POINT AVERAGE 



Clfissif Ication 

P»K)G«AM/SERV,CE ^SmAU SOPHC^QRE .UN. OR S£N,C« "cREKrER 

Mal« Library 



Undergraduate 
LitN-ary 

Car««r Planning t, 
Placamant Can tar 

Coun sating 
Cantar 

"•craatloft or 

Intramural 

Activity 

Writing Lab 

Studant 
Coip I oymant 



2.1 
3.4 
I.I 
1.4 
2.5 
1.2 



2.3 
3.4 

M 
1.3 
2.5 
I.J 



I 



2.6 
3.4 
1.3 
1.3 
2.5 
1.5 



2.9« 
3.3 
1.8* 
1.2 
2.5 
1.1 



Sarvlcas 


1.3 


1.5 


1.4 


1.4 


Haalth Sarvlcas 


1.7 


2.0 


2.0 


2.2« 


Catipus Plays 


2.0 


2.1 


1.9 


2.2 


Campus Fllffl Sarlas 


2.1 


2..i 


2.2 


2.3 


Campus Concart 
Sarlas 


t.9 


2.1 


2.1 


2.1 


Computar Sarvlcas 


1.4 


1.9 


1.9 


2.1« 


University 
Sookstore 


3.6 


3.V 


3.6 


3.7 


N 


171 


190 


187 


237 



Grada 



THAN 



2.7 

3.3 

1.4 

1.2 

2.4 

1.0 

1.4 

1.9 
2.2 
2.4 
2.1 

1.9 

3.7 
344 



Higher means Indicate greater use 
* P < .05 
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SERVICE/PROGR/W 

>teln Library 

Und«rgr«!uat« 
Library 



C«r««r Planning 4 
PlacamanT C«nTe- 

Counsal ing 
Cantar 

RacraatJon or 

Intramural 

Activity 

Writing Lab 

Studant 
Ewploymant 

Haalth Sarvicas 

Campus Plays 

C««pus Film sarlas 

Campus Concart 
Sari as 

Computar Sarvicas 

t*» I varsity 
Bookstora 

N-809 



TABLE 4 

REASON FOR NOT USING SERVICE MORE ' 



HO FURTHER 
NEED 
FOR 
SERVICE 



80S 

34X 

36S 

78J 

38J 

52J 

69% 
78J 
77* 

67S 
6IS 

88{ 



NOT 
FAMILIAR 

WITH 
SERVICES 



010 NOT 
HAVE 
NEEDED 
SERVICE 



FOUND 
SERVICES 
TO BE LOU 
QUALITY 




5S 

53J 
53J 
161 

54J 

32J 
9i 

I2S 
29% 



I2S 

9% 

6% 

5% 

1% 

IIS 

6S 
8J 
8S 

I7J 
9J 

5J 



2S 
4S 
4S 

5S 
IS 
2% 
4S 



16J 
3J 
1% 

2J 
6S 



• ^-antagas c.p„ta. on.y ,or stu.nts using t.at s.rvlce 
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TABLE 5 

RATING OF QUALITY AND IMPORTANCE OF SERVICES 



SERVICE/PROGRAM 



I M«lB Library 

Undergraduate 
Library 

f Career Planning i 
Placement Center 

f Counseling 
fCenter 

I.Hecreation or 
'Intramural 
fAetlvlty 

|Mriting Lab 

istudent 
inployment 
iervices 



University 
tookstore 



«>809 



Rating 

EXCELLENT GOOD FAIR 



POOR 



24 J 

25J 
16J 
lit 

m 

12> 



48? 

55J 
42J 
40S 

set 

22S 
36J 



26J 
18J 
31J 
37J 
22J 
48J 
45J 



3t 

3S 
US 
12t 

2f 
J 31 

71 



26S 



521 



I8J 



4S 



Importance. 
MEAN 



2.7 
2.8 
2.2 
2.0 
1.7 
1.6 
2.0 



.Health Services 


20t 


36J 


28S 


171 


2.3 


1!empus Plays 


35X 


48J 


15J 


2S 


1.7 


Campus Fl im Series 


22S 


54 J 


21t 


41 


1.6 


Campus Concert 














25S 


45J 


27J 


3S 


1.6 


Computer Services 


12S 


38J 


39J 


lis 


2.2 



2.3 



Percentages computed only tor students using that service 

Higher means Indicate greater Importance of that service to the student - Three point scale 
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TABLE 6 



PROGRAM/SERVICE UNIVERSITY ^S^* ^^^^ ^^1- Col. Col, Col. 



H 



' wlbrory 

IMdtrgroduote 
Library* 

C«r##r Planning 4 
PUc«nj»nt Canttr 

Counseling 

^•cr«otion or 

Intramural 
Activity 

>*rltlng tab 

Studant 

Enployfflmt 

S«rvtca$ 



2.9 

3.0 

2.6 

2.5 

2.9 

2.4 

2.5 



H*«»th Services 
Campus Plays 

C^Pus Fn« Series 2.9 

Campus Concert 
Ser Us 

Computer Services 

University 
Bookstore • 

K 

809 



2.9 3.1 
2.9 3.1 

2.5 2.8 

2.6 2.4 
2.9 2.9 

2.7 2.1 
2.5 2.6 



2.9 

3.0 

2.6 

2.6 

3.1 

2.5 

2.4 



2.9 3. I 

170 J18 



2.9 

68 



3.0 

3.0 

2.8 

2.7 

2.9 

2.5 

2.7 



2.6 


2.5 


2.8 


2.5 


2.6 


3.2 


3.2 


3.1 


3.2 


3.0 


2.9 


3.0 


2.9 


3.0 


2.9 


2.9 


3.0 


2.8 


3.0 


2.7 


2.5 


2.5 


2.5 


2.7 


2.6 



3.0 

86 



2.9 


2.8 


3.0 


2.8 


2.8 


3.0 


2.7 


3.0 


3.3 


2.9 


2.7 




2.? 


2.6 


2.4 


2.5 


2.1 


2.7 


2.4 


2.3 


3.1 


> 


3.2 


3.0 


2.6 


2.7 




3.5 


2.3 


2.4 


2.4 


2.5 


2.6 


2.7 


2.0 


2.4 


2.5 


2.5 


2.8 


2.5 


3.3 


3.1 


3.3 


3.1 


3.0 


2.8 


2.8 


3.1 


2.8 


3.0 



"•9^.r «e,„s .ncct. ..gher rating serv.c. 
• P < .05 



2.8 2.9 

2.3 2.6 

3.0 2.6 

72 62 



3.0 
2.4 

3.2 

73 



2.9 
2.5 

3.3 

65 



2.7 
2.4 

2.8 

83 



14 Q 
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SERVICE 

Admissions* 

Registration 

Student Records 

Student Loans 

Studant Grants 

Scholarships 

5 Student Conduct 
I Office 

Treasurer ' s 
^Office 

Orop/Add Procedure 
)PreregistratJoo 
N 



UNIVERSITY 



3.0 

2.4 

2,9 

2.3 

2.3 

2.5 

2.5 

2.5 
2.2 
2.9 

809 




TABLE 9 

STUDENT RATINGS Of UNIVERSITY SERVICES- 
MEANS Sr COUEGE 

Mean Responses * 
Col. Col. Col. Col. Col. Col. 
ABC T) V V 



3.0 

2.4 

2.8 

2.4 

2.4 

2.4 



3,0 

2.4 

2.9 

2.2 

2.2 

2.4 



3.0 

2.5 

3.0 

2.6 

2.3 

2.6 



3.0 
2.5 

t 

3.1 
2.4 
2.4 

2.5 



2.8 

2.3 

2.8 

2.3 

2.3 

2.8 



2.7 

2.2 

2.8 

2.2 

2.t 

2.0 



Col, 
G 



2.9 

2.4 

2.9 

2.2 

2.2 

2.3 



2.5 


2.4 


2.6 


2.6 


2.5 


2.1 


2.5 


2.4 


2.6 


2.6 


2.8 


2.5 


2.4 


2.6 


2.2 


2,1 


2.2 


2.3 


2.0 


2.2 


2.3 


2.9 


2.9 


3.0 


3.0 


3.0 


2.6 


3.1 


170 


lie 


68 


86 


72 


62 


73 



Col. 
H 



3.0 

2.4 

3.0 

2.3 

2.3 

2.5 

2.3 

2.6 
2.3 
3.1 

55 



Col. 
I 



2.7 

2.2 

2.8 

2.2 

2.1 

2.5 

2.4 

2.6 
2.2 
2.7 

83 
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TABLE to 

STUDENT RATINGS Of UNIVERSITY SERVICES- 
MEANS BY CLASS AM) GRADE POINT AVeSgE 



Maan Rtsponses 

Classification 
FRES^MN SOPHOMORE JUNIOR SENIOR 



Grade Point Averana 
GREATER THAN 3.0 LESS THAN 3.0 



' Higher means Indicate higher rating 



Admissions 


3.0 


2.9 


2.9 


3.0 


2.9 


3.0 


Registration 


2.5 


2.3 


2.4 


2.3 


2.4 


2.4 


Student Records 


2.9 


2.8 


2.9 


3.0 


2.9 


2.9 


oTuuvfiT Loons 


2.2 


2.3 


2.2 


2.4 


2.3 


2.4 


Student Grants 


2.3 


2.4 


2.1 


2.4 


2.4 


2.3 


Scholarships 


2.7 


2.4 


2.3 


2.4 


2.5 


2.4 


Student Conduct 


2.5 












Office 




2.6 


2.4 


2.4 


2.4 


2.5 


Treasurer's 














Office 


2.4 


2.4 


2.5 


2.8 


2.7 


2.5 


Orop/Add Procedure 


2.3 


2.0 


2.1 


2.2 


2.1 


2.2 


Pr er eg 1 strati on 
N 


3.0 


2.8 


3.0 


3.0 


3.0 


2.9 


171 


190 


187 


237 


344 


462 




j 
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TABLE 7 




I W'OGRAM/SEfiVJCe 



*teln Library 

Uf>d«rgradoat« 
library 

Caratr Planning a 
PlacMwtt Can tar 

Ooufisaling 

J" 

:''«er««tlon or 

UntrMurai 

^Activity 

'Iting Lab 

itudant 
•ploymant 
"•rvlcas 

•••ltd Services 

"PUS Plays 

•PUS film s«rl«s 

^ "PUS Concart 
irlas 

■>utar Sarvicas 

ivcrslty 
Okstora 



i!ii£lRo$pon$as' 
Classification 

FRESmAH SOPhCNORE 



3.0 
3.2 
2.6 

2.7 

3.0 

2.4 

2.7 

2.9 
3.2 

3.0 



^'EATE^ THAN 3.0 LESS THAN 3.0 



2.a 

3.0 
2.4 
2.3 
2.9 

2.4 

2.5 



2.9 

2.8 

2.7 

2.9 

2.8 

2.5 



2.9* 

2.7 

2.4 

2.9 

2.3 

2.4 



2.5 


2.5 


2.6« 


3.1 


3.1 


3.2 


2.9 


2.8 


3.0 



2.9 

3.0 

2.8 

2.6 

2.9 

2.3 

2.6 

2.6 
3.3 
3.1 



2.9 
3.0 
2,5* 

2.5 

2.9 

2.5 

2.5 

2.6 
3.l« 
2.8« 




Hfl^T «.ans Inc. cata higher ratl„ 
< .05 



9 of service 



15 
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SERVICE 

Admissions 

Rtglstration 

Student Records 

Student Loons 

Student Grants 

Scholarships 

Student Conduct 
OHice 

Treasurer's 
Office 

Orop/Add Procedure 
Prereglstratlon 
N»e09 



TABLE 6 

STUDENT RATINGS^ OF UNIVERSITY SFftvirFc. 
QUALITY AfO CLARITY OF^^SeS 



Rating of Servic* 
EXCELLENT GOOD FAIR pooft 



2IS 


561 


I9J 


i% 


9S 


38X 


33J 


20S 


24S 


50? 


2U 


6$ 


IIS 


321 


US 


2i% 


I4f 


29S 


34J 


24 J 


17X 


321 


3IJ 


21? 



I6S 



36J 



29? 



20S 



I5J 


41? 


29S 


I6J 


lOX 


2S% 


3?J 


31? 


26J 


47J 


21J 


6S 



' P.rc.nt«g,s ccput.. oo.y for stuOaors using that servlc. 



Clarity of Procedtjr<»>^ 



SOMEWHAT 

CLEAR CONFUSING CONFUSING 



56X 

32J 

61? 

251 

25J 

32S 

52$ 

55? 
42J 
74J 



38S 
411 

32J 
42J 
39J 
39J 

32J 

30J 
36J 
22S 



6S 
21% 

It 
33X 
36J 
29J 

I6J 
22J 
5> 



IS 



ERIC 



--I 



i 

TABLE II 

|. HAJi^ OF OUA tlTr AM) IMPORTANCE OF PR0C5RAMS AND SERVICES IN THE MAJOR 



MAJOR 2Sll£l Iwportano. 

I SERVICE/PROGRAM EXCELLENT GOOD FAIR POCR " ^^^^ 

5? " . 



Avai laDi i Ity of 
Advisor 



731 431 221 
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Ml 1 1 ingn«ss of 

i ^^^^^o^^o^^P 341 34J 24J 8J 

I Owtllty of Printed 

Program Information 131 40J 32J i3j 



Helpfulness of the 
Office Staff 

Quel Ity of Special 
Events 



201 461 25% 7% 

381 301 14J 



'71 481 281 



7% 



Adequacy of Prepar 
at ion by lover )42 
Division Courses for ^ 
Upper Division Courses 

Quality of Courses for 

Providing General £duc. 2I| 55| 2M 3J 

9ueilty of Courses In 

Preparing for Employment laj 47 j 29J 6j 

Avaiiabillty of 
* Required Courses for I3j 
the Major ^ 23jt 2.7 

Availability of 

Desired Courses for ]2X xi;« 

the Major }9% 2.5 

0rganl2atioo of the 
: Curriculum 



2.4 

2.5 
2.2 
2.0 
1.9 
2.6 

2.5 
2.8 



2.4 
2.4 



pirness of Grading ,3, 55, 261 6< 

i^^ality of Instruction 

Vln lover Division ,3j .c* 
^Courses in the Major * 2.6 

i(^ality of Instruction 

|la Upper Division xt* .n- 

^Courses In the Major 2.7 
'^portunlties for inter 

.action wltn Faculty In 201 xa* 

|he Major ^ 29 J 15J 2.2 

fVactlcum or Internship 

25J 30S 271 ,»j 

iibrary Col Uctlon 
UU, to th. Major ,e, ,5, 31 

■iri>ao9 ~~ 



2.4 

2.2 



|B.SM on students r.porting » «»jor or int.n.d.d major 

|H.9..r «.ns Indict, gr.at.r Importance ot t.,t service to t., student - T.ree point sca.e 

134 




TABLE 12 



MAJOR 



S£R V I CE/PROGRAM UN I VER S I T Y 



Col. Col. Col. Col. Col. Col. 

ABC n 17 ^ 



Col, 
G 



AvdHabllity of 
Advisor* 

Mi i ( ingn^is Of 
Advisor to H»lp« 



2.8 



2.9 



Ou«nty Printed 
Progrw Information* 2.6 

^•Ipfulntss of th# 
Offi€# St6ff* 2.8 

Quality of Sp«cidi 
Events* 2.6 

Adequacy of Pfptr 
at loo by Loirtr 2 8 

Division Courses for * 
Upper Division Courses* 

Quality of Courses: 2.9 
Providing General £duc.« 

Q^idllty o# Courses: 2.8 
Prej>arlng tor Employ t.« 

Availability of 
Required C&yrses 2 4 
for the Major* 

Avaitabil Ity of 
Desired Courses for 2 4 
the Major* 

Organization of the 2 8 
Curriculum* 

Fdlrness of &-adlng* 2.7 

Quality of instruction 
In low*r Olvlslon 2 6 
Courses in ^he Major* 

^al Ity of Instruction 
In Upper Division 3.1 
Courses In the Major 

Opportunities for inter 
action with faculty 2.6 
In the Major* 

PractlciM / Intern 2 6 
In the Major* 

Library Collection 2 7 
Related to the Major** 



Col. 
H 



2»8 2.8 3 2 o c , ^ 

^•2 2.5 3.0 2.7 2.9 2.7 

2.9 3.5 2.7 3.1 2.7 



3.0 3.0 



2.e 2.. 3.0 2.6 

2.8 3.2 2.8 2.9 2.4 



3.0 2.8 



2.8 3.0 2.9 2.6 

2.7 2.4 3.0 2.7 

2.6 ,.2 3.. 

2.6 2.6 2,9 2.6 3.3 2.9 

2.4 2.7 3.0 2.7 

2.5 2.5 2.9 2.7 
2.7 2.3 3., 2.8 
3.0 2.4 2.7 2.8 
2.7 2.6 2.8 3.0 

3.0 3.. 3.3 3.2 



2-5 2.5 3.2 2.4 2.9 2.8 
2-3 2.3 2.7 2.7 3.2 2.0 
2.7 3.0 2.6 2.7 2.3 



3.0 2.8 

3.1 3.2 
3.0 3.1 



TABLE 13 

RATING OF QUALITY OF PROGRAMS AM) SERVICES IN THE MAJOR: 
MEANS' CLASS AND GRADE POINT AVERAGE 



I SERVICE 



Classification 
FREShf4AN SOPHOMORE JUNIOR SENIOR 



Gradt Point Avtraqt 
GREATER THAN 3.0 LESS THAN 3.0 



'i Availability of 

I Advisor 2.7 

I Wllllngnass of 

I^Advltor to Hilp 2.9 

f Quality of Printtd 
if'Frogrm Information 2.7 

Iv^Haipfutnass of tha 
feOffIca Staff 2.8 

H 

> Quality of Spaclal 
feEvants 2.6 

Adaquacy of Pr^ptr 
I at loo by Lowar 2.6 
t Division Coursas for 
|Uppar Division Coursas 

J^Quallty of Coursas for 
l^^^rovldlng Ganaral Educ. 2.9 

[Quality of Coursas In 
^Praparlng for Emp^oy'r 2.8 

fAvi^l lability of 

^R'itioir0d Coursai lor 2.6 

i^tfia Major 

lAval lability of 

Oaslrad Coursas for 2.5 
tDa Major 

fOrganizatlon of the 
^Curriculum 2.8 

iFalrnass of Grading 2.8 

{Quality of Instruction 

|tn towar Division 2.6 

Oaursas in tha Major 

|: 

^Quality of Instruction 

*ln Uppar Division 2.8 

Coursas In tha Major 

•Jrv 

(Spportunitlas for lnt«r 
ibtion with Faculty In 
,Jh« Major 

f 

{■ractlcun or Intsrnship 
i«p.rl«nr»$ - th« Major 

library Col I act I on 
■ I .tad to tha Major 



2.4 

2.7 

3.0 

171 



2.6 
2.8 
2.5 
2.7 
2.5 
2.8 

3.0 

2.8 
2.2 

2.2 

2.7 
2.6 

2.6 
3.0 
2.4 

2.5 

2.6 

190 



2.7 
3.0 
2.5 
2.7 
2.7 
2.8 

2.9 
2.8 

2.5 

2.3 

2.8 
2.8 

2.6 
3.0 
2.7 

2.7 

2.7 

187 



2.9» 

3.0 

2.6 

2.9* 

2.6 

2.8 

3.0 
2.7 
2.5* 

2.5» 

2.7 
2.8» 

2.6 

3.2« 

2.8* 

2.6 

2.7* 

237 



2.8 
2.9 
2.6 
2.9 
2.7 
2.8 

3.0 
2.8 
7.i 

2.5 

2.8 
2.8 

2.7 
3.1 
2.7 

2.7 

2.7 
344 



2.7 

3.0 

2.6 

2.8 

2.5« 

2.8 

2.9 
2.8 
2.3* 

2.3« 

2.7 
2.7« 

2.6 
3.0 
2.6 

2.6 

2.7 
462 



/KIghar «aans Indlcata highar rating 



lERJC 



IfSG 



. . .JBPP^^^^ 



I 

I: 

f 



TABLE U 

RATING OF QUALITY AfC ^^^^AJCE^OF^F^^^ OF THE CUSSROO, EXPERIENCE. 



145 



EXCELLENT 



Ratings ^ 
GOOD FAIR 



POOR 



Cofflprehcn*ncss-» 
of Course Content 

Relevenct of Content 
for Student Needs 

Extent to which 
Content Is Current 

Instructor's 
Class Presentations 

Instructor's 
Class Preparation 

Instructor's Enthu- 
siasm for Teaching 
this Class 

Instructor's Helpful- 
ness /Student Problems 

Fairness of Testing 

Fairness of Grading 

Clarity of Course 
Objectives 

CondueIven«s$ of 

CI imate for Learning 

Relevance of Lecture 
Information to 
Course Objectives 

Quality of 

Classroom Discussion 

Accuracy of Catalog 
Description • Course 

Instructor's Knowledge 
of Subject Matter 

Instructor's 
Aval labi I Ity for 
Consultation 

Overall Qual ity of 
Instructor 

Overall Qual Ity of 
Course 



241 
37S 

26S 
15S 
66S 
32S 

48S 
24S 



40J 
41J 

32J 
51S 
26% 
23S 

31S 
46S 



29% 

m 

26% 
26% 
1% 
18S 

14J 
2A% 



8S 
61 

m 

8S 

8S 

8J 
61 



N»809 



Importance^ 



MEAN 



26% 


50J 


J9J 


51 


— 

2.3 


24t 




24> 




2.3 


39J 


4 5< 


XAt. 


2% 


2.3 


37J 


35J 




11J 


2.5 


47J 


33J 


»5J 


51 


2.5 


531 


27J 


13J 


7J 


2.4 


441 


291 


19J 


9$ 


2.6 


30J 


431 


19J 




2.5 


12% 


431 


1 91 


71 


2.5 


24« 


42J 


25? 


ICS 


2.2 



2.2 
2.3 

2.2 
1.8 
2.7 
2.4 



, 6as*d on student evaluation of a specific randomly chosen course 

2 Higher «eans Indicate greater importance of that service to the student - Three 



point scale 



157 



I 
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TABLE 15 

RATING OF QUALITY OF FACETS OF THE CLASSROOM EXPERIENCE- 
MEANS BY COLLET ^ 



Maan R«spons«s^ 

UNIVERSITY ^f' ^^1- ^^1- Col, Col, 

ABC D E F 



Cofflpr»h»n<n»$$* 

of Courst Content 3,0 

Rtltv«nct of Content 
for Student Naeds 2.8 

Extent to vhlch 
Content Is Current 3.2 

Instructor's Class 
Presentations 3,0 

Instructor fs Class 
Preparation 3,2 

Instructor's Enthu- 
tiasa for Teaching 3,3 
this Class 

Instructor's Help 3.1 
Wltn Problems 

Fairness • Testing 2.9 

• Fairness - Grading 3.0 

Clerlty of Course 
Objectives 2.8 

^Conduclveness of 
CI Iwate-Learnlng 2.8 

Relevance of lecture 

information to 3, i 
5 Course Objectives 

T^atlty of Class- 2.7 
;room Discussion 

t- 

'Accuracy of Catalog 2.7 
Oascriptlon^Course 

^1nst*s Knovlsdg* 

^» Subjsct M8tt«r 3.6 

Jlnstructor's 
Availability for 3.0 
Consultation 



3.0 2.9 

2.8 2.8 

3.2 3.1 

2.9 3.0 

3.3 3.2 
3.3 3.2 
3.1 3.0 

3.0 3.0 

3.0 3.0 

2.8 2.9 

2.8 2.7 

3.1 3.0 

2.6 2.6 

2.7 2.7 

3.6 3.5 

3.1 3.0 



3.1 

2.9 

3.3 

3.0 

3.3 

3.3 

3.0 

2.9 
3.0 

2.8 

2.8 

3.2 

2.8 

2.7 

3.6 
3.0 



Ovaraii Quality of 
Ihstructor 

Ovarali Quality of 
teursa 



3.2 



3.2 3.2 3.3 



2.9 

2.8 

3.1 

2.8 

3.1 

3.2 

3.0 

2.9 
2.9 

2.7 

2.8 

3.1 

2.6 

2.8 

3.5 
2.8 

3.0 



2.9 3.1 

2.8 3.0 
3.1 3.3 

2.9 3.1 
3.1 3.2 
3.1 3.4 
3.0 3.3 

2.7 3.1 

2.8 3.1 

2.8 2.8 

2.5 2.9 
3.0 3.1 

2.6 2.8 

2.7 2.6 

3.J 3.6 

2.9 2.9 

3.0 3.4 



Col. 
G 



3.0 

2.9 

3.2 

3.0 

3,1 

3.1 

3.0 

2.9 
2.9 

2.9 

2.7 

3.1 

2.7 

2.8 

3.6 
3.0 



Col. 
H 



3.0 

3.0 

3.4 

3.0 

3.3 

3.3 

3.1 

2.8 
2.9 

2.8 

2.8 

3.1 

2.8 

2.6 

3.7 
3.1 



Col. 
I 



3.2 3.2 



2.8 

2.7 

3.1 

3.0 

3.3 

3.3 

2.9 

2.9 
2.9 

2.7 

2.6 

3.0 

2.6 

2.7 

3.5 
3.0 

3.1 




IiErIc 



1^8 
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I 

i 



i 



RATING 



TABLE 16 



OF OUAUTY AND IMPORTAfCF oc 



I 



Compr#h#n»n#$$- 
o* Courst Content 

R«l«v#nc« of Content 
tor Student Nta^s 

Exttnt to which 
Conttnt is Currtnt 



3.3 
3.2 



3.0 
2.7 
3.1 

InstTMctor's 

CU»$ PlTMMtations 3.0 

Instructor's 
Class Prtparation 

Instructor's Enthu 
this Class ^ 

I Instructor's Halpfui- 
I "•*» /Student ft-obll, 3 , 

I Palrnass of Tasting 3., 
j ''••mass of a-a<,|„g 3 , 

? X>J«ctivas - „ 

ontSuc Ivanas:; of 
llMta for uarning 2.7 

•••vanca of Lactura 

iforaatfon to , „ 

'ursa Objactlvas 

lallty of 

assroom Discussion 

I curacy of Catalog 
^ scrlption - Course 

structor's Knowledge 
Subject Matter 

•true tor's 
; 'liability for 
'Switation 

•■a 'I Quality of 
tructor 

; -all Quality of 
-$e 




Class If let 1 ^ 



2.9 
2.7 
3.2 



2.9 
2.9 
3.2 



3.0 

2.9» 
3.3 



2.9 


2.9 


3.0 


3.2 


3.2 


3.3 


3.2 


3.2 


3.2 


3.1 


3.0 


3.1 


2.9 


2.8 


2.9 


2.9 


2.9 


3.0 


2.6 


2.7 


2.9« 


2.7 


2.8 


2.8 


3.0 


3.1 


3.2 



2.7 


2.5 


2.6 


2.8 


2.8 


2.7 


2.7 


2.7 


3.5 


3.5 


3.6 


3.5 


3.0 


2.9 


2.9 


3.l» 



I 9h.r ,eans Indicate higher rating 
< .05 



3.2 


3.1 


3.1 


3.3» 


2.9 
J7? 


2.8 

190 


2.9 
187 


2.9 

237 



I5i 



o 



3.0 
2.8 
3.2 
3.0 
3.3 
3.3 

3.1 
3.0 
3.1 

2.9 

2.8 

3.2 

2.7 
2.8 
3.6 

3.0 

3.2 

2.9 
344 



2.8 

3.2 

2.9 

3.2 

3.2 

3.0 

2.9* 

2.9» 



2.7 
2.7 
3.0» 

2.7 

2.7 
3.5» 
3.0 

3.2 

2.9 

462 



I 
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TABLE 17 



RATING OF OUAUTY AND IMPORTANCE OF FACETS OF THE ClASSROC;^^ EXPERIENCE: 
MEANS* BY FACUITY/GRAO ASST AND REASON FOR TAKING CLASS 



Instructcr 

Faculty Graduate 
Hamb«r Assistant 



Raason for Taking Class 

Major Raccomandad 
Ra<?ulr9mtnt For Major Elactlva 



|f Co«pr aDan • nass* 
U\Qf Coursa Contant 

f 

|rR#i«vanca of Contant 
|v for Studant Naads 

|j Extant to which 
|?Contant Is Currant 

if. 

Instructor's 
^^Class Prasantatlons 



Instructor's 
f Class fVaparation 



t 

Instructor* s Enthu- 
|>:Slasm for Taaching 
^ this Class 

Instructor's HatpfuU 
|:n9ss /Studant Problams 

|'f«lrnass of Tasting 

^falrnass of Grading 
fr- 
ies arity of Coursa 
lObJactivas 

jlConducivanass of 
IfCi inata for Laarning 

^, 

|R*lavanca of Lecture 
ll^lii format Ion to 
1|Caurse Objectives 

[iQuality of 
||Cla5sroom Discussion 

^Accuracy of Catalog 
llipescription • Coursa 

llfnstructor's Knowledge 
rot Subject Matter 

llinstruc tor's 
|;Avai lability for 
[jCoosultation 

1- 

lOverall Quality of 
lllnstructor 

lOyerall Quel ity of 
|(C#oursa 

k 

IN 



2.8 
2.9 
3.2 
3.0 
3.3 
3.3 

3.0 
3.0 
3.0 

2.8 

2.8 

3.1 

2.7 
2.7 
3.7 
3.0 

3.2 

2.9 
524 



3.0« 

2.8 

3.1 

2.9 

3.1 

3.1« 

3.1 

2.8« 

2.9 

2.7 

2.7 

3.0* 

2.5 
2.7 
3.3« 
3.0 

3.1« 

2.8« 

180 



2.9 
2.8 
3.1 

2. ? 

3. ! 
3.2 

3.C 
2.9 
2.9 

2.7 

2.7 

3.0 

2.6 
2.7 
3.5 
2.9 

3.1 

2.8 
477 



2.9 
2.9 
3.3 
2.9 
3.2 
3.3 

3.0 
2.9 
3,0 

2.8 

2.8 

3.1 

2.8 
2.7 
3.6 
3.1 

3.2 

3.0 

107 



3.1 

2.8 

3.3« 

3.2« 

3.5« 

3.4« 

3.3« 

3.0 

3.2* 

2.9« 

3.0* 

3.2 

2,9« 
2.7 
3.6 
3.1 

3.4« 

3.0« 

199 



P Higher maans indicate higher rating 
P < .05 



IerJc 



ISO 



i • 



lASit je 

0. INK A...J Ci!A..A:UK.bllCS: CL /.ws far C01116E 



Chorcdtr Istic 



SEX 
file 
(<■ eie 

I I SIOLN'CE 
r>L,rir. iiory 

ri.-.^.T.ity 
lone 

CC-r;ui.'ITY OF ORIGIN 
Form 

»3n-fdrm hural 
Tc>.n U.idcr 2, MO 
Tov.n 7/^00 - 7i,000 
TcMi 2i,000 - 100,000 
Cify Over 100,000 

iiOJ^S KCRKIK3 

> - 9 Ibors 
10 - J9 l-.ours 
20 or over h-.ors 

PROPORTION OF FOU'^A- 

TiON PAID Br ^A^£^.Ts 

»;ore Tf.c.T half 
los; Than hal f 
l.'ine 

NU.VSER OF ClOSf RBLA- 
TlO'.rniPS KITH FACULTY 

'.jne 

One 

Two 

Three or More 

HOl'RS ON CAMPUS EACH 
*>tCK OUTSIDE Of CLASS 

Under 10 
10 - 20 

20 - 30 

Over 30 



FRATERNITY OR SORORITY? 
Yes 
No 

SATISFIED WITH SOCIAL 
EXPERiENCE AT U.T.K7 
Very Satis< led 
Scuewhat Satisfied 
Somewhat Dissatisfied 
Very Dissatisfied 

c^lll'il^^ ACADEMIC 
EXPERIE^^CE AT U.T K7 
Very Satisfied 
Saxewhat Satisfied 
Scr.ewhat Dissatisfied 
Very Dissatisfied 



Univ. 



52? 



5J 

nj 

ICS 
24 f 
<2J 



54J 

7.7% 



59X 



Col. Col. Col. Coi: Col. Col. Col. Col. 



Col. 

I 



22? 



29? 
KJ 
8? 
49? 



22J 

78? 



36? 
48? 
13? 
4? 



20? 
52? 
23? 
5? 



'e? 



<2S 
i'5? 
5? 

>e? 



5? 
9? 
2? 
13? 

'''6? 
<5? 



53? 
8? 

le? 

21? 



62? 
13? 

25? 



14? 

23? 
23? 



36? 
46? 
14? 
"? 



15? 
51? 
26$ 
7? 



61? 

i&? 



<2? 
29? 
<? 



3? 

7? 

1? 
21? 
21? 



50? 

9? 
13? 

2B% 



65? 


fc'? 


12? 


35? 


16? 


bb% 


31? 


5i? 


52? 






28? 


9? 


4X> 


0? 


26? 


i-i? 


20? 



41? 
«? 
12? 
3? 



24? 
50? 
21? 
5? 



27? 


5? 


11? 


24? 


12? 


10? 


3? 


4? 


3? 


4? 


19? 


19? 


19? 
^-2? 


24? 

ib? 


21? 
36? 



46? 

15? 

le? 

22? 



65? 
12? 
9? 
14? 



6ej 


52? 


<e? 


10? 


24? 


26? 


21? 


24? 


26? 


55? 


35? 


69? 


12? 


9? 


12? 


10? 


21? 


lot 


23? 


35? 


10? 



31? 


27? 


29? 


14? 


14? 


19? 


7? 


9J 


13? 


"7? 


50? 


38? 


22? 


35? 


29? 


78? 


65? 


71? 



41? 
'!4? 
10? 
<? 



16? 
51? 
29? 
3? 



21? 
17? 
4? 
59? 



13? 
87? 



25? 
60? 
13? 
2? 



17? 
58? 
21? 
4? 



60? 

7? 
15? 

le? 



67? 
17? 
17? 



/!4? 

13? 
19? 
24? 



22? 
17? 
14? 
48? 



26? 
74? 



34? 
49? 
16? 
1? 



22? 
56? 
17? 
6? 



eo? 

20? 



52? 
35? 

4? 

9? 



5? 
15? 

5? 
11? 

26? 
39? 



63? 
10? 

le? 

ID? 



63? 
17? 
20? 



4 7? 
16? 
10? 
27? 



7? 

13? 
11? 
69? 



13? 
67? 



29? 
47? 
19? 
5? 



21? 

55? 
23? 
2? 



e? 

92? 



51? 
26? 

0? 
24? 



7? 
Ifc? 

6? 
14? 
19? 
37? 



66? 

7? 
13? 
13? 



58? 
15? 
26? 



47? 

7? 
23? 
23? 



3S? 
10? 
8? 
44? 



28? 
72? 



52? 
38? 

6? 

4? 



36? 
"9? 
12? 
3? 



23? 

77? 



47? 

20? 

1? 
22? 



2? 
11? 

8? 
17? 

:;5? 
■'If 



57? 
9? 
14? 
20? 



57? 
11? 
32? 



37? 
16? 
25? 
22? 



4 5? 

9? 

5? 
42? 



12? 
88? 



36? 

9? 
5? 



32? 
40? 
23? 
5? 



35? 
05? 



4 0? 
27? 
2? 
31? 



AS 

17? 
2? 
13? 

:<% 

-•9? 



40? 

7? 
22? 
31? 



63? 
19? 
16? 



20? 
14? 
20? 



34? 
12? 
11? 
43? 



28? 
72? 



41? 
43? 
16? 
0? 



'6? 
46? 
32? 
6? 



Rir 



APPENDIX 9 

Instructional Evaluation Information 
for College D 



Prepared for 
College D 
by 

The Learning Research Center 
1819 Andy riolt Avenue 
Telephone 974-2459 
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Sources of Instructional Evaluation Information for College D 



NOTE: (+) or (-) Based on University Average 
(&-) » Lowest rating of all Colleges 



JII. STUDENT RATINGS 



I. STUDENT ACHIEVEMENT 

General Education 

Mean Entering ACT Score (+) 
Mean Total CC^P Score (+) 
Estimated Gain on COMP (-) 

Achievement in Major 

National Professional Exam 

II, OTHER SOURCES 

Dean's Follow-Up Survey of 
Seniors (Winter 1984) 

Student Evaluations of 
Instruction 



ERIC 



Quality of 
Program Services 
in the Major 
Table 12 

Availability of A r\ \ 
A4vi3or 

Wlltlngntss of (-) 
Advisor to hoi p 



(-) 



Qual ity of Printad 
Proqrsn Information 

Helpfulness of the 
Office Staff 

Qual i ty of Special 
£vents (-) 



Adequacy of Prepar 
etion by tower 
Division Courses for 
Upper Division Courses 

(>jallty of Courses: 
Frovtvilng General Educ 

Qual Ity of Courses: 
Preparing for L-nployt* 



Aval tabi I t ty of 
Required Courses 
for the Major 



(-) 



Avaltabi t ity of 
Desired Courses for (-) 
the Major 

Crgsnization of the . 
Curriculum C"*/ 

Fairness of Q-adIng (-) 

Quality of Instruction 
In Lower Division (©-) 
Courses in the Major 

^allty of Instruction* 
In upper DIvl sion (-) 
Courses in the Major 

Opportunltlos for Inter 
action with Faculty (Q-) 
In the Major 

Pracrlcujn / Intern (+) 
in the Major 

Library Col lection (-) 
Related to The .^jor 



Quality of 
Classroom 
Experience 
Table 15 



Comprehend ness« 
of Course Contend 



.(-) II 



Relevence of Content 



for Student Needs 

Extent to vhlch (-.) 
Content Is Current 



Instructor's Class 
Presentations 



Instructor's Ctas 
Preparation 



Instructor's Enthg* 
slas« for Teaching (-) 
this Class % 



1 



Instructor's Help 
KItn Problems (-) 



1 



Fairness • Testing ^ 



fairness - Grading (-) I 



Clerlty of Course / \ ^ 
Objectives 



Conduci veness of 
CI Imare^Learnlng 

Relevance of Lecture 

Information to 
Course Objectives 

^al ity of Class- (_) 
room Discussion 

Accuracy of Catalog (+) 
Description-Course 

Inst's Knowledge / \ 
of Subject Matter 

Instructor' s 
Aval labi I Ity for (©-) 
Consultation 

Overal I C<Jal ity of . . 
Instructor (") 



- 1 



Overal I C^al Ity of 
Course (-) 



Satisfaction with Social Experience at OTK (4-) 
Satisfaction with Academic Experience at OTK <+) 



3 

1 

I 

I 

I 
1 



a. 
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lOU 



95 



90 



85 



80 



75 



f I 
t i 
+ 1 
t I 
+ 1 
i 1 
+ j 

I i 
+ 1 
n 
+1 
II 
+ 1- 



70 


+ j 




1 1 


65 


+ 1 






60 


+ f)0 




1 1 > 


55 


■*"**\ 




1 j5A^ 


50 


+ 1 — 




1 t 


4*5 


+ 1 




1 1 


40 


*l 




I 1 


35 


+2 




t 1 


30 


+ 1 




1 i 


25 








20 


+ 1 




li 


15 


+ 1 




1 1 


10 


+ 1 




11 


5 






1 1 


0 


♦1 




i 1 




1 1 
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I 

I 

I 
1 

1 • 
I 
1 
I 
I 
I 
I 

I 
I 
I 
I 

■^1 
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?SI 




I 

I 
1 
1 
I 
I 

I 

■I- 
1 
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Seniors in College D 
I I 

Seniors • 
I 

( 
I 
I 
I 
I 
I 
I 

+ 

COW • 



— 
UA 



— -t- 
S? 



40TK: 



1 Obb UluOt-:; 



:>uuTi.br:^ 1-6 Aai) total oCOui: 



164 



•-.11 
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University of Tennessee-KnOKville 
COMP Objective Test Means (Equated to Fora III) 
for Senior Samples Tested in 1980-1933 





May 1980 
N " 165 Seniors 


April 1981 
N « 680 Seniors N 


April 1982 
■ 644 Seniors 


Mav 1983 
N « 700 Seniors 


local 


192.3 


187.9 


189.3 


188.9 


ro J. 


66. 1 


65,1 


64.1 


63.5 


us 


66.1 


64.0 


64.3 




UA 


60.1 


59.1 


61.2 


61.3 


COM 


55.8 


54.7 


54.3 


54.0 


SP 


78.3 


76.5 


76.7 


76.6 


CV 


58.2 


56.8 


58.2 


58.2 


ACr/E 


23.6 




21.5 


22.3 


(N) 


(119) 




(467) 


(382) 






COMP SCALES 







ERIC 



• Communicating: Can send and receive information in a variety of 
modes (written, graphic, oral, numeric, and symbolic), within a 
variety of settings (one-to-one. in small and large groups), and for 
a variety of purposes (for example, to inform, to understand, to 
persuade, and to analyze). 

• 

• Solving Problems: Can analyze a variety of problems (for exam- 
ple, scientific, social, personal); select or create solutions to 
problems; and implement solutions. 

• Clarifying Values: Can identify one's personal values and the 
personal values of other individuals; understand how personal 
values develop; and analyze the implications of decisions made on 
the basis of personally held values. 

• Functioning within Social Institutions: Can identify those activ iJes 
and institutions which constitute the social aspects of a culture (for 
example, governmental and economic systems, religion, marital 
and familial institutions, employment, and civic volunteer and 
recreational organizations); understand the impact that social 
institutions have on individuals in a culture; and analyze one's own 
and others* personal functioning within social institutions. 

• Using Science and Technology: Can identify those activities and 
products which constitute the scientific/ technological aspects of a 
culture (for example, transportation, housing, energy, processed 
food, clothing, health maintenance, entertainment and recreation, 
mood-altering, national defense, communication, and data pro- 
cessing); understand the impact of such activities and products on 
the individuals and the physical environment in a culture; and 
analyze the uses of technological products in a culture and 'one*s 
personal use of such products. 

• Using the .Arts: Can identify those activities and products which 
CG:.oatute the artistic aspects of a culture (for example, graphic art, 
music, drama, literature, dance, sculpture, film, architecture); 
understand the impact that art, in its various forms, has on 
individuals in a culture;'and analyze uses of works of art within a 
culture and one's personal use of art* 



1 



240 
80 
80 
80 
72 

72 



1$5 



University of Tennessee^ Knoxville 
COMP Estimated Mean Gains by College 
Based on Seniors with ACT Scores 



1982 



1983 



College 


N/ 


Mean 


Estimated 


N/ 


Mean 


Estimate 




ACT 


ACT 


Mean 


APT 


APT 


Mean 




Score 


Composite 


Gain on 


Score 


Composite 


Gain on 








COMP 






COMP 


A 


32 


22.1 


7.2 


29 




22.1 


9.3 


B 


8 


26.0 


6.4 


17 


23.8 


4.1 


C 


142 


21.3 


9.0 


lOl 


21.7 


6.1 


E 


34 


19.9 


9.0 


35 


20.9 


10.1 


F 


42 


18.2 


13.2 






11 1 




56 


25.6 


2.5 


78 


24.3 


1.7 


G 


63 


19.8 


7.5 


19 


20.3 


7.2 


H 


38 


23.8 


4.2 


65 


23.4 


2.4 


I 


53 


20.3 


11.5 


13 


23.0 


4.8 


All Combined 


468 


21.5 


8.1 


382 


22.3 


5.2 
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Percentages of Students Having Taken College Mathematics 




in 



IJnlver- 



College Algebra 
or Pre-Calculus 

Business 
Calculus 

Mathematics^ of 
Finance 

Freshman 
Calculus I 

Freshman 
Calculus II 

Freshman 
Calculus II.I 

Sophomore 
CalcuJus I 

Sophomore 
Calculus II 

Sophomore 
Calculus III 



I erIc 



A3. 6 



39.0 



35.8 



27.8 



27.0 



26.3 



Col. 

A 



27.7 58.3 



25.8 56.3 



27.1 



16.7 



10.4 



8.3 



A. 2 



4.2 



Col. 

D 



Col. 

C 



217 



60.9 



52.2 



4.3 



0.0 



0>0 



21.5 



20.9 



6.4 



5.8 



4.1 



Col. 
E 



59.3 14.3 



0.0 



0.0 



0.0 



Col. 

F 



56.3 83.3 26.1 79.7 63.3 41.7 



30.4 56.4 30.6 6.9 



5.6 



65.2 28.5 10.2 15.3 



8.2 13.9 



4.1 12.5 



6.9 



6.9 



6.9 



Col. 
D 



N = 148 



Col. 

G 



0.0 



0.0 



Col. 

H 



73.3 '49.7 



20.0 19.0 



16.7 16.3 



10.0 46.3 



10.0 39.5 



0.0 32.7 



0.0 21;8 



21.1 



19.7 



Col. 
I 



81.5 
0.0 
0.0 

18.5 
7.4 
3.7 
0.0 
0.0 

0.0 
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♦I 


p 




11 




00 




d 






C 


5b 


+b-~ 


ji 




II 


A 


bO 




1' 
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Percentages of Students Having Taken 2- or 3- Course Sequences in Natural Sciences 



Astronomy 
Basic 

Engineering 

Biology 

Botany 

Chemistry 

Geological 
Science 

Microbiology 

Physics 

Zoology 



Univer- 
sity 

7.8 



18.6 
36.0 
3.6 
45.5 

14.1 
3.8 

33.5 
8.5 



Col. 

A 

0.0 

10.4 
64.6 
16.7 
89.6 

18.8 
10.4 
33.3 
8.3 



Col. 
B 



0.0 

8.7 
8.7 
0.0 
13.0 

0.0 
4.3 
91.3 
0.0 



Col. 

_JL- 

12.8 



2.3 
31.4 

1.2 
12.8 

18.0 
1.2 

11.6 
1.7 



Col. 
E 

16.3 

0.0 
26.5 

0.0 
12.2 

26.5 
0.0 
0.0 
0.0 



Col. 

F 

15.3 



2.8 
70.8 

2.8 
22.2 

23.6 
1.4 

15.3 
6.9 




Col. 

H 

8.2 



4.1 
40.8 

7.5 
36.7 

12.9 
3.4 
25.2 
12.9 
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Percentages of Students Having Taken the Following Social/Applied Sciences 



Univer- Col* Col. Col* Col- Col. Col. Col* 

sity A B _C_ E F D 



Anthropology 


31.3 


33.3 


8.7 


15.7 


63.3 


37.5 


15.5 


50.0 


Aiidiology 6f 
Speech Patho— 
logy 


4.1 


4.2 


0.0 


1.2 


4.1 


19.4 


0.0 


0.0 


Economics 


71. 1 


85.4 


1714 


94.2 


95.9 


40.3 


82.4 


93.3 


Geography 


33.8 


37.5 


4.3 


36.6 


32.7 


48.6 


42.6 


16.7 


Human Services/ 
Social Work 


11.0 


0.0 


0.0 


11.6 


6.1 


34.7 


2.0 


16.7 


Political Science 


41.3 


27.1 


4.3 


47.7 


98.0 


43.1 


23.6 


26.7 


Psychology 


74.9 


75.0 


56.5 


71.5 


95.9 


97.2 


66.2 


83.3 


Sociology 


61.6 


56.3 


21.7 


64.0 


95.9 


79.2 


48.0 


76.7 
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Percentages of Students Having Taken the Following Areas In a History Sequence of at Least Two Courjses 




Univer- 
sity 



Col. 

A 



Col. 
B 



Col. 
C 



Col. 

. E 



Col. 

F 



Col. 
D 



Col. 

G 



Col. 
H 



Col. 

I 



Afro-American 






























History 


1. 


3 


2.1 


0. 


0 


0.6 


0 


.0 


0.0 


0.7 


0.0 


4 


.1 


0.0 


American History 


36. 


2 


33.3 


26. 


1 


36.0 


20 


A 


58.3 


21.6 


33.3 


52 


.4 


14.8 


Cultural Studies 


3. 


A 


2.1 


0. 


0 


1.7 


2 


.0 


0.0 


1.4 


6.7 


8 


.8 


7.4 


Western Civiliz. ; 


21 


.8 


10. A 


13. 


0 


11.6 


95 


.9 


13.9 


6.8 


6.7 


36 


.7 


18.5 


World Civilization 


2 


.8 


2.1 


8. 


7 


1 .2 


2 


.0 


1.4 


4.7 


0.0 


4 


.1 


0.0 



1^5 
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Percentages of Students Having Taken the Following Areas of the Humanities 



Univcr- Col. Col. Col* Col* Col* Col. Col* Col* 

sity A B Q F. IL D „ G H 



Art - History 
or Appreciation 


22.5 


12.5 


39.1 


13.4 


20.4 


40.3 


15.5 


• 40.0 


32.0 


Art - Studio 


12.2 


12.5 


52.2 


5.8 


20.4 


9.7 


2.0 


23.3 


20.4 


Dance 


12.4 


16.7 


8.7 


10.5 


12.2 


25.0 


4.7 


16.7 


15.0 


Literature - Mod-"r» 
ern or Classical 


63.4 


37.5 


13.0 


73.8 


81.6 


73.6 


47.3 


70.0 


76.9 


Music - History or 


28.6 


14.6 


4.3 


19.8 


46 .9 


63. S' 


22.3 


40.0 


29.3 


Music - Perfor- 
mance 


10.2 


8.3 


4.3 


9.3 


16.3 


19.4 


4.7 


6.7 


11.6 


Philosophy 


38.0 


33.3 


26.1 


27i9 


44.9 


45.8 


26.4 


46.7 


49.7 


Religious Studies 


22.6 


6.3 


8.7 


14.5 


30.6 


33.3 


16.9 


33.3 


33.3 


Speech & Theatre 


44.8 


58.3 


8.7 


55.8 


79.6 


55.6 


23.0 


50.0 


42.2 
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TA8LE 2 



STUDENT USE OF GENERAL UNIVERSITY PRCG^MS AND SERVICES: 
MEANS BY COUUEG£ 











Mean Resoonses' 










PROGRAM/SERVICE 


UNIVERSITY 


Col* 
A 


Col. 

B 


Col. 
C 


Col. 
D 


Col. 
E 


Col. 

F 


Col. 

G 


Col. 
H 


Col. 
I 


Main library* 


2.5 


2.5 


2.5 


2.3 


2.4"* 


2.3 


3.0 


2.8 


2.3 


2,6 • 


Undargraduate 
Library* 


3.4 


3.5 


3.5 






•« •« 
^. ^ 


3.0 


3.0 


3.6 


3.4 


Carter Planning & 
Plactment Canter* 


1.4 


1.3 


1.4 


1.4 


1.5^ 


1.4 


1.2 


1.1 


hi 


1.2 


Counsal Ing 
C«nT«r* 


1.3 


1.3. 


1.5 


1.3 


i.r 


1.2 


1.2 


1.3 


bi 


1.2 


Racreatlon or 
1 nTranur a 1 
Activity* 


7 5 


* . *♦ 


9 7 


2.9 


2.7^ 


2.2 




2.4 


2.4 


2.4 


Writing lab 


1.1 


I.I 


1.1 


1.1 


1.1 


1.1 


1.1 


l.I 


1.2 


1.1 


Student 

Employnent 

Services 


1.4 


1.4 


1.3 


1 

1 . J 


+ 
1 C 
i . J 




1.4 


1.4 


1.3 


1.4 


Health Services 


2.0 


2.0 


2.0 




7 n 


7 7 


2.0 


2.4 


2.0 


2.1 


Campus Plays* 


2.1 


2.2 


1.9 


1.9 




2.2 


2.0 


2.0 


2.0 


2,5 


Campus Fl im Series' 


' 2.3 


2.4 


2.1 


2.2 


2.4^ 


2.1 


2.5 


2.1 


1.8 


2.3 


Campus Concert 
Series 


2.1 


2.2 


1.9 


2.1 




2.2 


2.0 


1.9 


2.0 


2.1 


Computer Services* 


1.8 


1.6 


2.4 


1.7 


++ 

2.5 


1.2 


2.4 


1.1 


1.2 


1.3 


Untverstt/ 
BooKstore 


3.7 


3.7 


3.6 


3.6 


3.7 


3.7 


3.7 


3.8 


3.7 


3.7 


N 


809 


170 


tl8 


68 


86 


72 


62 


73 


65 


83 



Higher means Indicate greater use 
• P < .05 



NOTE: ++ Highest of all colleges 

+ Higher than University Mean 
- Lower than University Mean 
0- Lowest of all colleges 
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TABLE 6 

PERCEIVED QUALITY OF GENERAL UNIVERSITY PROGfWWS AND SERVICES- 

MEANS BY COLLEGE 



Mean Responses^ 



PROGRAM/SERVICE 



Col. Col. 
UNIVERSITY A B 



Col. 

C 



Col. 
D 



Cbl. Col. 

E F 



Col. 

G 



Col. 
H 



Higher means Indicate higher rating of service 
• p < .05 



Col. 
I 



Hdin Library 




0 Q 


1 


2.9 


3.0 


2.9 


2.8 


3.0 


2.8 


2.? 


Undergraduate 
LJ brary* 






















3.0 


2.9 


3.1 


3.0 


3.0 


3.0 


2.7 


3.0 


5 T 
J* J 




Career Planning & 






















PiacoQenT Center 






2.8 


2.6 


2^8 


2.7 


2.3 


2.9 


2.6 


2.4 


Counsel 1 ng 






















Center 


2.5 


2.6 


2.4 


2.6 


2.7^ 


2.5 


2.1 


hi 


2.4 


2.3 


Recreation or 






















Intramural 
Act |y 1 ty 


2.9 


2.9 


2.9 


3.1 


2.9 


3.1 


2.9 


3.2 


?.0 


2.6 


Writing Lab 




2, 7 


2, 1 


2.5 


« -+ 
2.5 


2.7 


1.5 


3.5 


2.3 


2.4 


Student 






















Employment 
Serv ices 






2.0 


2.4 


2.7 


2.4 


2.5 


2.6 


2._7 


2.0 


Health Services 


7 6 






2.3 


2.6 


2.4 


2.5 


2.5 


2.8 


2.5 


Campus Plays 


3.2 


3.2 


3.1 


3.2 


3.0" 


3.3 


3.1 


3.3 


3.1 


hi 


Campus Film Series 


2.9 


3.0 


2.9 


3.0 


2.9 


2.8 


2.8 


3.1 


2.8 


3.0 


Campus Concert 






















Series 


2.9 


3.0 


2.8 


3.0 


2.7- 


2.8 


2.9 


3.0 


2.9 


2.7 


Computer Services 


2.5 


2.5 


2.5 


2.7 


2.6-^ 


2.3 


2.6 


2.4 


2.5 


2.4 


Un I versi ty 






















Bookstore 


3.0 


2.9 


3.1 


2.9 


3.0 


3.0 


2.6 


3.2 


3.3 


2.8 


N 
t 


809 


170 


tl8 


68 


86 


72 


62 


73 


65 


83 
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TABLE 9 

STUDENT RATINGS OF UNIVERSITY SERVICES: 
MEANS BY COLLEGE 



Mean Responses 
Col. Col. Col. Col. tJol. Col. 



Col. 



Col. Col. 



\ SERVICE UNIVERSITY 

>^ 


A 


B 


C 


D 


E 


F 


G 


H 


I 


\ 

Admissions* 


3.0 


3.0 


3.0 


3.0 


3.0 


2.3 


2.7 


2.9 


3.0 


2.7 


; Registration 

i 


2*4 




















Student Records 






















2.9 






\ n 


T 1 


7 ft 


T A 


7 0 




7 ft 


Student Loans 


2.3 




7 7 


^ fi 
« •« 


7 d 


7 




*•* 




7 7 


* Student Grants 


2.3 




2.2 


2.3 


2.4* 


2.3 


2.1 


2.2 


2.3 


2.1 


^ Scholarships 


2.5 


2.4 


2.4 


2.6 


2.5 


2.8 


2.0 


2.3 


2.5 


2.5 


Student Conduct 






















Office 


2.5 


2.5 


2.4 


2^ 


2.6^ 


2.5 


2.1 


2.5 


2.3 


2.4 


Treasurer's 










4+ 












Office 


2.5 


2.4 


2.6 


2.6 


2.3 


2.5 


2.4 


2.6 


2.6 


2.6 


Orop/Add Procedure 


2.2 


2.2 


2.1 


2.2 


2.3* 


2.0 


2.2 


2j3 


2^ 


2.2 


Preregi strati on 


2.9 


2.9 


2.9 


3.0 


3.0^ 


3.0 


2.6 


3.1 


3.1 


2,7 


N 


809 


t70 


118 


68 


86 


72 


62 


73 


65 


83 



Higher means Indicate higher rating 
• P < .05 
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TABLE 12 

RATING OF QUALITY OF FT^OGRAWS AfO ScRyiCES IN 
MEANS BY COLLEGE 



THE MAJOR: 



Mean Responses 



SPPVir^/Snro.. Col. Col- Col- 

SERVICS/rSCGRAM UNIVERSITY A B C D E 



Col. 

F 



Col* 

G 



Col. Col. 
H I 



Avai rabi f ity of 
Advisor* 2,8 

WI I { Ingness of 

Advisor TO Help* 2.9 

C««l «ty of Printed 
Program Jnfor-nar lon« 2.6 

Helpfulness of the 
Office Staff* 2.8 

Ouallty of Special 
• Events* 2.6 

Adequacy of Prepar 
ation by lower 2.3 
Division Courses for 
Upper Division Courses* 

Qual ity of Courses: 2.9 
Providing General £duc.» 

Quality of Courses: 2.8 
Preparing for E-nployt,* 

Aval tab! I ity of 
Required Courses 2.4 
for the Major* 

Aval labt I Ity of 

Desired Courses for 2.4 

t^e Major* 

C^ganization of the 2.8 
Curriculum* 

Fairness of Q-adIng* 2.7 

Quality of Instruction 
In lower Division 2.6 
Courses in the Major* 

^al Ity of Instruction 
In Upper Division 3.1 
Courses in the Major 

Opportunities for Inter 
action with Faculty 2.6 
In the Major* 

Practlcum / intern 2.6 
In the Major* 

Libra-y Col lection 2.7 
Related to the Major* 

N« 809 



2.8 2.8 

3.0 2.9 

2.7 2.5 

2.3 2.8 

2jj5 2.6 

2.3 2.8 

3.0 2.9 

2.7 2.8 

2.5 1.9 

2.5 2.1 

2.8 2.8 
2.8 2.7 
2.7 2.5 

3.0 3.i 

2.5 2.5 

2.3 2.3 

2.7 2.7 

170 US 



3.2 
3.5 
2.6 
3.2 
2.7 
2.8 

3.0 
2.8 

2.6 

2.6 
2.8 
3.0 
2.7 

3.2 

3.2 
2.7 
3.0 

68 



^ Higher means indicate higher rating 



2.7' 
2.4- 
2.8 

2.5' 
2.8 

2.8" 
2.8 

2.2" 

2.3" 
2.7- 
2.6" 

&" 
3.0" 

2.7+ 
2.6" 

86 



3.0 
3.1 
2.8 
2.9 
2.8 
2.7 

3.0 
2.9 



2.3 
2.7 
3.0 
2.7 



2.9 
3.2 
2.7 

72 



* P < .05 

182 



2.7 

izl 
2.1 

2.4 
3.0 
2.4 

2.6 
2.6 



2.4 2.7 



2.5 
2.3 
2.4 
2.6 



3.0 3.1 



2.8 
2.0 
2.3 

62 



2.9 
3.0 
3.0 
5.0 
2.9 
3.0 

3.2 
3.3 

-3.0 

2.9 
3.1 
2.7 
2.8 

3.3 

3.0 
3.1 
3.0 

73 



2.7 2.7 
3.0 3.0 
2.6 2.7 

2.8 2.8 

2.6 2.8 

2.7 2.5 

3.) 2.8 

2.9 2.5 

2.7 2.2 
2.7 

2.8 2.6 . 
2.8 2,5 

3.0 2.6 

3.2 3.0 

2.8 2.7- 

3.2 2.9 

3.1 2.7 

65 83 



^1 



41 

i 
1 

•ll 



11 
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TABLE 13 

RATING OF QUALiTY OF FACSTS OF THE ClASSROCH EXPESIEfCE- 
MEANS BY COLLEGE^ 











2 

Mean Resoonses 












UNIV£=^SITY 


Col* 

A 


Col, 
B 


Col. 

C 


Col. 

D 


Col. 

E 


Col. 

F 


Col. 

G 


Col. 
H 


Col. 
I 


Ccmprahen iness- 
of Course Content 


3.0 


3.0 


2 9 


3.1 


2.9" 


2.9 


3. 1 


3.0 


3.0 


2.8 


Relevence of Content 
for Stuoent .^eds 2.8 


2 8 


7 ft 


2.9 


2.8 


2.8 


3.0 


2.9 


3.0 


2.7 


Extent to which 
Content Is Current 


3.2 • 


3.2 


3.1 


3.3 


3.1" 


3.1 


3.3 


3.2 


3.4 


3. 1 


Instructor's Class 
Presentations 


3.0 




















2.9 


3.0 


3.0 




2.9 


3,1 


3.0 


3.0 


3.0 . 


Instructor's Class 
Preparation 


3.2 


J •J 




3.3 


3.1~ 


3.1 


3.2 


3.1 


3.3 


3.3 


Instructor's Enthu- 
slaso for Teacning 3^3 
this Class 


3.3 


3.2 


3.3 


3.2" 


3.1 


3.4 


3.1 


3.3 


3.3 


Instructor's Help 
WItn Prooiems 


3.1 


3.1 


3.0 


3.0 


3.0" 


3.0 


3.3 


3.0 


3.1 


2.9 


Fairness - Testing 


2.9 


3.0 


3.0 


2.9 


2.9 


2 7 


3.1 


2.9 


2.8 


2.9 


Fairness - Grading 


3.0 


3.0 


3.0 


3.0 


2.9" 


2.3 


3.1 


2.9 


2.9 


2.9 


Clarity of Course 
Obj ecti ves 


2.8 


2.8 


2.9 


2.8 


2.7- 


2.8 


2.8 


2.9 


2.8 


2.7 


Conduc i veness of 
CI Imate-Uearn Ing 


2.8 


2.8 


2.7 


2.8 


2.8 


2.5 


2.9 


2.7 


2.8 


2.6 


Relevance of Lecture 
Information to 3, l 
Cocrsa Objectives 


3.1 


3.0 


3.2 


3.1 


3.0 


3.1 


3.1 


3.1 


3.0 


Cual Ity of Class- 
room Discussion 


2.7 


2.6 


2.6 


2.8 


2.6" 


2.6 


2.8 


2.7 


2.8 


2.6 


Accuracy of Catalog 2.7 
rescript Ion-Course 


2.7 


2.7 


2.7 


2.8^ 


2.7 


2.6 


2.8 


2.6 


2.7 


Inst's Knowledge 
of Subject Matter 


3.6 


3.6 


3.5 


3.6 


3.5" 


3.4 


3.6 


3.6 


3.7 


3.5 


Instructor* s 
Availabil Ity for 
Consultation 


3.0 


3.1 


3.0 


3.0 




2.9 


2.9 


3.0 


3.1 


3.0 


Overal 1 Qiial ity of 
Instructor 


3.2 


3.2 


3.2 


3.3 


3.0" 


3.0 


3.4 


3.2 


3.2 


3.1 


Overal 1 Qua! Ity of 
Course 


2.9 


2.9 


2.9 


3.0 


2.8~ 


2.8 


3.0 


2.9 


3.0 


2.8 


H 


809 


170 


118 


68 


86 


72 


62 


73 


65 


83 



College in vhlch course is b«lng tal^en 
Higner means Indicate higher rating 
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StieCleO STUDLNT OPINIO'. S Ai.O OiARACl LRl bl ICS: I'.':anS BY COLLEGE 



CharoClerliT Ic 



Uolv. 



Col. 

A 



Col. 
B 



Col. 



Col. 
D 



Col. 

E 



Col. 

F 



Col. 

G 



Col. 
H 



Col. 
I 



SEX 
I'! la 

r.iSIOLNCE 
r-ui in i 1 ory 
t'-ai 'i <unT 
fi n I ty 
li^-ae 

a--.-'Ji,'|TY OP ORIGIN 
form 

K'3n-(orm Rural 
Ten Under 2, ^00 
Town 2,500 - 25,000 
Tc»n 35,000 - 100,000 
City Over 100,000 

nOU'.S KCRKIt.'G 
(.'one 

1 - 9 lours 
10 - 19 l ours 
20 or ever K->urs 

PSOPCSTION OF KDUCA- 
TICH PAID ar f-AKEKTS 

(•'ore Then ha I f 

Less Thin hal t 

Uzne 

NU:;3ER OF CLOSE RELA- 
TlO-'-'ilPS KITH FACULTY 

'One 

One 

T»o 

Three or l-lore 

Hoyas ON c;k?us each 

K££K OursiOE OF CLASS 
Under 10 
10 - 20 
20 - 30 
Over 30 



FRATERNITY OR SORORITY? 
Yes 
No 

SATISFIED WITH SOCIAL 
EXPERIENCE AT U.T.K7 
Very Satisfied 
So.-newhat Satisfied 
Sor.ewhat Dissatisfied 
Very Dissatlsf ied 

SATISFIED WITH ACADEMIC 
EXPERIENCE AT U.T.K7 
Very Satisfied 
Saxewhat Satisf led 
Somewhat Dissatisfied 
Very DissAtisf led 



52J 
4SJ 



3J 

>6> 



it 
lis 
3J 

16J 
2ii 
<2S 



54J 

Mi 
22S 



59J 
151 
26J 



MS 
22S 



29i 
A9i 



22J 

m 



36J 

13J 
4J 



20J 
52J 

23J 
5? 



i2J 



4 2J 
28? 



5J 
9X 
2< 

13J 
26J 
<5J 



53J 

es 

2J< 



411 
MJ 
231 
23J 



31J 
47J 



22J 

m 



36S 

46$ 
14$ 
4$ 



15$ 
51$ 
26$ 
7$ 



61$ 

39$ 



42$ 
29$ 
4$ 
2:$ 



3$ 
7$ 
1$ 

21$ 
21$ 
43$ 



50$ 

9$ 
13$ 

23$ 



62$ 68$ 

13$ lo; 

25$ 21$ 



55$ 
12$ 
10$ 
23$ 



27$ 
14$ 
9$ 
50$ 



35$ 
65$ 



41$ 

44$ 
12$ 



24$ 
50J 
21$ 
5$ 



ti$ 
i5$ 



31$ 

9$ 
26$ 



27$ 
24$ 
3$ 
4$ 

19$ 
22$ 



46$ 
15$ 
Id? 
22$ 



52$ 
24$ 
24$ 



35$ 
9$ 
21$ 
35$ 



29$ 
19$ 
13$ 
38$ 



29$ 
71$ 



41$ 

44f 
lOX 
4$ 



16$ 

51$ 
29$ 
3$ 



B7$ 
16$ 



55$ 

;'8$ 

4$' 
14$ 



5$ 
12$ 

4$ 
19$ 
24$ 
3S$ 



65$+ 
12$ 

14$ 



48$ 
26$ 
26$ 




21$- 
17$ 
4$ 

59$-»- 



13$ 
87$+- 



25$ 
60$-t+ 
13$ 
2$- 



17$- 
58$+ 
21$ 
4$- 



12$ 

b3$ 



52$ 

25% 
0$ 

20$ 



11$ 
10$ 
3$ 
19$ 
21$ 
36$ 



60$ 
7$ 
15$ 
18$ 



67$ 
17$ 
17$ 



14$ 
13$ 
195 
24$ 



22$ 
17$ 
!4$ 
48$ 



26$ 
74$ 



34$ 
49$ 
16$ 
1$ 



22$ 
56$ 
17$ 



80$ 
20$ 



52$ 

3S$ 
4$ 
9$ 



5$ 
15$ 

5$ 
11$ 

26$ 
39$ 



63$ 
10$ 

1S$ 
10$ 



63$ 
17$ 
20$ 



47$ 
16$ 
10$ 
27$ 



7$ 
13$ 
11$ 
69$ 



13$ 
67$ 



29$ 
47$ 
19$ 
5$ 



21$ 
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23$ 
2$ 



8$ 
92$ 



51$ 

?6$ 
0$ 
24$ 



7$ 
18$ 

6$ 
14$ 
19$ 
37$ 



66$ 
7$ 
13$ 
13$ 



58$ 
15$ 
26$ 



47$ 
7$ 
23$ 
23$ 



33$ 
10$ 
8$ 
44$ 



28$ 
72$ 



52$ 
38$ 
6$ 
4$ 



36$ 

49$ 
12$ 
3$ 



23$ 

77$ 



47J 
?0$ 
1$ 
72$ 



2$ 
11$ 

8$ 
17$ 
35$ 
'3$ 



57$ 
• 9$ 
14$ 
20$ 



57$ 
11$ 
32$ 



37$ 
16$ 
25$ 
22$ 



45$ 
9$ 
5$ 

^2$ 



12$ 
88$ 



36$ 
50$ 
9$ 
5$ 



32$ 
40$ 
23$ 
5$ 



35J 
05$ 



40$ 
27$ 
2$ 
31$ 



17$ 
2$ 
13$ 
24$ 
-9$ 



40$ 
7$ 
22$ 
31$ 



63$ 
19$ 
16$ 



46$ 
20$ 
14$ 
20? 



54$ 
12$ 
15$ 
43$ 



28$ 
72$ 



41$ 
43$. 
16$ 
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16$ 
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UNIVERSITY or itJif^ESSEE 
STUDENT SATISFACTION Br COLLEGE 
1983 SURVEY 
LEARfilNO RESEARCH CENTER 

In Spring of 1983, a survey of undergraduate of the Unlverilty of TtnnastM, KnoKvlll«^ 
undartaken by tha Learning Ramrch Cantar. Ovar 2000 undargraduatca vara quarlad In an attai^ 
Isolate opinions around three underlying foci; the unlveralty as a whole, the colleges, i 
selected departsients. Reports have been forwarded to appropriate persotmel at each of t 
levels, and so«e decisions have been wade In response to these reports. At the Uilversltyn 
level, the responses have been clearcut; some problems with services provided to, or Mde avail 
to all students, have been corrected. At the department level, there have likewise been < 
adjuslments, ranging from Improvement of written program Information to Intense evaluation 
Individual Instructors and the use of teaching assistants. However, the essencm of our fMti 
at least from the standpoint of effecting meaningful policy ImprovmMnt Is ptf*obably to be foiflli 
the col lege level. We found much variation among tha col lege on a nimber of toplciw Wiaraal 
most meaningful variation In teaching Is to be found at the depertment level, with virtually 
differences among the colleges, there was significant variation in a nimibar of crucial dlMM^ 
«»ong the col leges. In particular, we found that the whole asblance of student-faculty Interact 
differed, from relatively close relationships In the College of Agriculture, to virtual sluf 
anonymity In Engineering. Not only did student satisfaction vary among the collegaa, but 
factors which lead to satisfaction likewise varied, with students fn some col logos Judging 1 
experiences In mostly academic terms, while others attached less Importance to ecedamlc cues» 
variation becomes Important when one real Izes that overal I feel Ings of satisfaction are rel 
student retention. In fact, these evaluations of the university experience are Just as li 
In determining the likelihood of students' returning as are their grado point avaragaiu 
policy-making perspective, than, the Import Is clear. Different strategies for maximizing 
tlon must be undertaken In each col lege. The dynamics of student opinion formation are diff 
these opinion are formed In the context of different stimuli, and these opinions matter. A 
synopsis fol Sows; 

t. Stodantst lovels of satisfaction with their experience at the University of Te 
vary by collage. These amounts are shown In Flgure-1. |t must be remembered that 
being measured here Is the students' reaction toward their entire experience at^ 
university, and not merely within their col leges. Satisfaction Is not a one dimensional 
that Is, It can be decomposed into social and academic components. The Important thing laj 
so much the levels of satisfaction felt by each of students <Flgure-2), but the detemla 
of these levels (Figure-!). 

2. The col leges vary In the degree to which academic and soclai satisfaction d4 
overall satisfaction. That Is, students' overall reactions to the university are 
product of their academic experiences In some col leges than In others. Conversely, the 
experience Is predominate In other situations. Overall satisfaction across the unlversH 
e whole Is more a product of academic than social factors, although the difference |s sll^ 
and Is not statistically significant. 

5. Students In the professional col leges. Nursing and Architecture and, to some da 
Business are somewhat unique In the way In which they react to their university experU 
They tend to base their conclusions on more wacademlc* factors than do other students, end] 
Interaction with faculty In the major seems particularly Important. 



4. Satisfaction Is related to actual student behavior. Students who are satisfied are*i 
likely to return to the university, even when the tact of their having lower grade polnti 
taken Into account. 
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Dftt«nil9iants of Sntisfoctfon by Go I \ 



AGRICULTURE 

Students in th« col lege of agrlcuf tur« ore fairly typical of UT students in t#nM of acadMic, 
social* and ovarall satlsfoction. Hovavar* that is In rafaranca to thair university axparlaneo* 
Thay are euch nora satisfied with their experiences In their college. But this college level 
feel Ing Is not enough to translate Into a nora favorable university experience* As FIgure-t Bhom%, 
social experiences outveigh acada»lc ones In accounting for the reactions of agriculture students* 
Vhen we look further into the covponents of these social and acadavlc eveluatlons^ there are soM 
lnterestl»)g relationships which are relevant to ail this. The ealn factor underlying acadaelc 
satisfaction In the college of agriculture is Not the experler ce with the feculty In the Mjor, but 
nore than in any other college, with the exception of education^ Is GPA. That ls» agriculture 
studenti^ feolings of acadavlc satisfaction are largely a product of hov well they ere doing, 
rather than their perceptions of the quality of academic programs* SInply put, then, there Is not 
much to be done to increase these feelings* On the other hand, agriculture students are unique In 
what accounts for their social satisfaction. Unlike the students In any other college, those In 
this unit are euch wore satisfied M they are a samber of a fraternity or sorority. Perheps It Is 
the physical Isolation of the agricultural campus from the core of campus life, hut th I s exper I encs 
Is a strong determinant of their overal 1 social feel ing, which Is highly related to their overel I 
reaction to the University of Tennessee* Thus students In the col lege of agriculture are happier 
with the atmosphere In their major, but no different than other UT students In hov they relate to 
the entire university* Social satisfaction Is Important to tham, and this seams to be a product of 
whel'h^r Ihey «*e In a Greek organization* 

ARCHITECTURE 

Those enrol In this col loge are a unique group* In many ways they are the most "'academic'* 
at the ijnty«r«<ty. Thay are slightly more satisfied with their academic experience et the unlver-^ 
slty, and sllghttv less satisfied with their social lives, than are students In other colleges* 
Howevar, It Is not so much the level of satisfaction, but the determinants of satisfaction, which 
are enlightening* Architecture students^ overa! I reactions to UT are much more a product of their 
academic experience than of their social experiences* tn this regard, they rely on their academic 
cues more than students In any other col isge* The key academic variable Is the degree If Interac- 
tion with the faculty In the major* To a large degree, architecture students ere social (y happy If 
they are academically pleased, but being Involved with a fraternity or sorority Is almost as 
crucial as for those In the coi lege of agriculture* It would seem as If these students feel Inge 
toward the university could be altered by encouraging more student-faculty Interaction* Unlike 
those In agriculture, GPA Is not that cruciai to academic satisfaction* 

BUSINESS 

Students In the college of business are somewhat happier with their social lives at the 
university than other students, and though no different In regard to their reaction to academics, 
are somewhat more positive In their overall assessments* It Is clear, thojgh from flgure-1 that 
they are somewhat more I Ikety to rely on academic cues In their overal I Judgments ebout UT* They 
are second to the architecture students In the degree to which their summary evaluations are a 
function of academic as opposed to social stimuli* This academic component Is e function of many 
sub-components* Unique to this group Is the Importance of a good relationship with the edvlsor; 
not as Important to the overal I Judgement of the student Is his/her evaluation of randomly selected 
clesses, most of which were In the college* Social satisfaction among business students Is 
affected both by Greek membership and by the number of students the respondents felt they knew 



1^7 



170 



w#l \. It would SMI that the col lege of business could maximize student feel Ing at the university 
by concentrating on th^e advising function and clossroo« perfonnance. Both are key factors In 
effecting student reaction to the University* 

EDUCATION 

Education students are quite typical of the student body as a whole In their ecedMilc, social, 
and overal I reactions to the university. They are also typical In the relative weights of social 
and acadenic factors In accounting for overall satisfaction. Here, as In agriculture, the ealn 
detemlnant of academic satisfaction Is the student^s GPA. There Is also seme tendency for the 
evaluation of the acadealc experience at the university to depend on the qual Ity of the advising In 
the major, and to the qua I Ity of classroom Instruction as measured by the students* responses to a 
randomly selected class. Social satisfaction Is largely a product of the number of close tfui^nf 
relationships enjoyed by the student. Thus It seems that the col lege of education could Increase 
student reaction by working on the advising system and encouraging more student Interaction outside 
of class. 



ENGINEERING 



Engineering students had the worst feelings about the quality of advising and felt the most 
distance from the faculty In their mc|or fields. Their overal I feel Ings about the university were 
not atypical, however. Students In this college do rely a more heavily on social cues than 
academic ones for the«r summary assessments. The only s+rong sub-component In the social area was 
that of student Interaction. Those with more relationships with other students were much more 
likely to have a better Impression of the social side of the University of Tennessee. On the other 
hand. Prater n i ty/Sorcr I ty membership bore no relationship to overall social satisfaction mong 
these students. (» the acadewilc side, the amount of Interaction with faculty In the major was a 
strong determinant of academic satisfaction. Given the low degree of such Interaction, It would 
seem that the col lege of engineering Is In a particularly good position to affect a positive change 
In attitude among its students If a way to Increase faculty-student Interaction outside of the 



classroom can be found. 



HOME ECONOM ICS 



Home economics student are a little less pleased with the social atmosphere at the university, 
and a little loss pleased with the academic environment. Is than the rest of the student body. 
Social and Academic factors are equal ly Important In determining overal I satisfaction, although the 
relationship Is a little less strong than In the university as a whoje. Academic satisfaction Is 
mainly a function of grade point average, while social satisfaction Is mainly a product of academic 
satisfaction. Simply put, students In this college are satisfied with their overall university 
experience If they are doing well, and while they have opinions about their programs, these fee- 
I Ings do not S0m that Important In forming their overal I Impressions of the university. 



LIBERAL ARTS 



While students In Liberal Arts are typical of university students as a whole, they are a 
little more pleased on the social dimension of campus lite, and a little less pleased with their 
academic experiences. Likewise, social phenomena are more Important than academic variables In 
accounting for the variation In overal I satisfaction that exists In the col lege. The wost critical 
component of social evaluation Is the nimiber of student relationships enjoyed by the student. 
Acedemic satisfaction throughout tfje col lege Is mainly a product of GPA. Here again, one should 
not expect that major change In student reaction could be brought about easily. Possibly, some 
attempt could be made to Involve students more with each other, especial ly since the type of In- 
volvement relevant here Is outside of the Fraternity/Sorority system. 
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COMMUNICATIONS 

Cowimlcatlons students or9 a Mttia Ims satlsfla^ with th#lr oy#r5l I eKperlence st 1ST fhsn 
avaraga studant, and alto a I Ittli* lass pSaasad vlth thair acadanic axparlanca. Hovavar thair 
suMary faallngs ara quita haavlly influancad by thalr social axparlaacas, aora so than for any 
othar group* Thalr acada«lc aval uat Ions ara, to a graat daal, a product of thalr gpA, and tharo* 
fora not subjact to «uch In tha way of an "iwprovawant stratagy. On tha othar hand, thara Is a 
vary s\rong tandancy for studants In this col laga to faal social ly satisfied If thay hava a nuibar 
of clota studant rat at lonsh I ps. This faal Ing Is aspaclal ly strong In this col laga. 

NURSING 

Nursing studants ara tha aost ptaasad with thalr axparlanca at tha Un I varsity, Tha^ ara also 
sacond only to archltactura In tha Inportanca of acadamlc avaluatlons to tha ovaral I raactlon to 
unlvarslty Ufa. Tha rain datamlnant of acadamlc happlnass saam to ba tha Intaractlon with 
faculty In tha »aJor, again. Ilka thosa In archltactura. Tha wain datanalnant of social satisfac- 
tion Is again tha nunbar of closa ratatlonshlps with othar stud*»nts. 



SATISFACTION AND RETENTION 

Of cour^a, tha util Ity of satisfaction as a concapt Is tiad to tha dagraa to which It af facts 
actual studant bahavlor. in ordar to ascartain this wa ra-axawlnad tha sanpla In tha Fal I of 1984 
to saa If aach raspondant had rmfurn^d to tha unlvarslty. Each cowponant of satisfaction, both 
acadsMlc and social was usad to try to account for tha ffQrn, or non«raturn, of tha studant, along 
with two othar varlablas which pro)f%d to ba ralatad to ratantlon»-CPA, and tha nwhar of hours 
workad. It was posslbia to analyza tha antira sampla In this «annar« Thus whila fft9r9 Is a 
good dagraa of avidanea that tha coaponants of satisfaction vary by col laga, tha luport of satis- 
faction wl { I only ba gaugad for tha who I a unlvarslty. 

Our findings for tha whola unlvarslty Indlcata that acadamlc satisfaction Is slightly »ora 
llkaly to affact a studants* likallhood of ratu^nlng to UT than Is social satisfaction. Vhan tha 
coMponants ara co»blnad, thay ara statistical ly significant In a iM>dal of ratantloo, along with a 
atudant^s gradapoint and tha nunbar of hours workad par waak« Of varlablas, satisfaction Is 

tha strongast pradlctor. it Is liaportant to nota this ralatlonship holds whan GPA Is taiam Into 
•ecount, statistically, or control lad for. Thus avan though It Is tha studants who hava highar 
avaragas who ara aora plaasad with thalr axparlancas at tha Uilvarslty, thara Is sowathlng unlqua 
about satisfaction apart frow tha dagraa to which it taps how wal I a studant Is doing* Thasa 
Judgaants tap soMthIng that Is strong anough to affact a studant*s daclslon to r^tiMrn to tha 
unlvarslty. Final ly, this daclslon to raturn Is also affactad by tha nuMbar of hours tha studant 
w>rks. Thosa who work ara lass llkaly to coaia back* Probably this is a function of financial 
hardship for aost of tha raspondants. 

In conclusion, dlssatlsflad studants, along with thosa who ara doing badly ara llkaly to laava 
tha unlvarslty, along with thosa who ara working. Each of thasa r9frt to a unlqua componant of tha 
studaot*s axparlanca. Thalr avaluatlons of what thay find ara cortalnly garaana to thalr bahavlor, 
that Is, what thay think of tha quality of thalr •npT\enco is avary b!t as laportant to than as 
how wal I thay do and how financial ly stralnad thay ara. 
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ncuRC 1 

OVERALL SATISFACTION AS AFFECTED 
BY ACADEMIC AND SOCIAL SATISFACTION 

rOR ALL STUDCNTS & BY COllCGt 
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SATISFACTION: 
OVERALL, ACADEMIC AND SOCIAL 

FOR ALL STUDENTS k BY COLLCCE 
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TABLE- 1 

Major Determinants of Academic Satisfaction ; By College 



College 



Interaction with faculty In the major 

- GPA 

- Interaction - Advisori GPA, Instruction 

- GPA Interaction with Advisor 

- Interaction with faculty In the major 



•Architecture 
Agrlcuture 
Bualoess 
Education 
Engineering 
Home Economics - GPA 
Liberal Arts - GPA 

Communications - GPA - Avalllty of Instruction 



Hurslng 



-Interaction with faculty In the major 



Major Determinants of Social Satisfaction : By College 



College 

Architecture 
Agriculture 
Business * 
Education 
Engineering 
Home Economics 
Liberal Arts 
Communl ca t 1 ons 
Nursing • 



- Greeki Academic Satisfaction 

- Greek 

- IClose Relationships/Students, Greek 

- Academic Satlsfactloni tfClose Relationship/Students 

- #Close Relationships/Students 

- Academic Satisfaction 

- j^Close Relationships/Students 

- #Close Relationships/Students, Greek 

- #Close Relationships/Students 
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TABLE 18 

SELECTEO STUDENT OPINIONS AND CHARACTERISTICS: McANS BY COtlEGE 



Characteristic 



SEX 
HaU 

Fuma 

RESIDENCE 
Dornltory 
Apartment 
Fratern I ty 
Hoai« 

COMMUNITY OF ORIGIN 
Fartt 

Mon**farm Rural 
Town Under 2,500 
Town 2,500 - 25,000 
Town 25,000 - 100,000 
City Over 100,000 

HOURS WORKING 
Mon« 

1 - 9 Hours 
10 - 19 Hours 
20 or over Hours 

PROPORTION OF EDUCA- 
TION PAID BY PARENTS 

More Than half 

less Than half 

None 

NUNBER OF CLOSE REU- 
TIONSHIPS WITH FACULTY 

None 

One 

Two 

Three or More 

HOURS ON CAMPUS EACH 
WEEK OUTSIDE OF CLASS 

Under 10 

10 - 20 

20 - 30 

Over 30 



FRATERNITY OR SORORITY? 
Yes 

SATISFIED WITH SOCIAL 
EXPERIENCE AT U.T.K7 
V«ry Satisfied 
Somewhat S<msf led 
SoMwhat Dissatisfied 
Vary Dissatisfied 

SATISFIED WITH ACADEMIC 
EXPERIENCE AT U.T.K7 
Vary Satisfied 
SoMwhat Satisfied 
SoMwhat DIssatlsf lad 
Vary Dissatisfied 
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13$ 
87$ 
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0$ 
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MODEL OF STUDENT RETENTION AT THE UNIVERSITY OF TENNESSEE 



OVERALL 
SATISFACTION 



HOURS 
WORKING 




RETENTION 



GRADE 
POINT 
AVERAGE 



CONCLUSIONS 



U The degree to which a student works has an Impact on his/her likelihood of returnlnq, with 
work ng eading to a decision not to return. This effect Is both direct and '"direct; that Is 
working jo*'«rs Gpa which lowers the likelihood of returning, and working also lowers 
satisfaction with the university, which likewise lowers the likelihood of returning. 

2. Students' CPA's are related to their retention also, with a lower CPA lessening the likelihood 
Of return, »s we I I as contributing to a lesser degree of satisfaction, which also has a direct 
Impact upon student retention. 

3. Overall ^s»^^\*»ct>on Is directi y re lated to retention, and Is the variable manifesting the 
largest direct effect. That Is, taking Into account whether a student works and his/her GPA 
satisfaction Is the most Important variable In determining whether a student returns. 

4. Though this model Is for the entire university, the determinants of overal I satisfaction vary b/ 
col lege. n any case, the one variable reflecting student opinion, or their reactions to what i 
they experience Is the most Important cue students use In making the decision to return. Whll« 
there s probably very little that can be done about reducing student needs to work, and nothino 

^ w Increase GPA, coNeges and departments do have some strategies ^ 

which can be pursued In order to effect a more satisfied student population. 

5. One must not overemphasize the degree to which massive changes are possible. We are only able \ 
to explain a smal I part of the variation In decisions to return, and many of the factors \ 
affecting these decisions are beyond anyone's control. However, those factors which are subject 
TO Improving are those which can be addressed In the context of Improving the Quality of ^ 
programs. a m ^ 

.6. This model Is based on the 1983 student satisfaction survey (General University Sample.. n«910) 
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APPENDIX 11 

RESULTS OF FRESHMAN CQMP TESTING 
University of Tennessee^ Knoxville 

Fall 1983 



Prepared by 

Learning Research Center 
University of Tennessee, Knoxville 
1819 Andy Holt Avenue 
Knoxville, TN 37996 
Telephone : 615-974- 2459 
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Results of FRESHMAN COMP Testing 
Fall 1983 



College 


Number 
Tested 


Mean 
ACT 
Composite 


CCWP 
Total 


TOTAL 
Percen- 
tile 
Rank 


FSI 
Percen- 
tile 
Rank 


US 
Percen- 
tile 
Rank 


UA 
Percen- 
tile 
Rank 


COM 
Percen- 
tile 
Rank 


SP 
Percen- 
tile 
Rank 


CV 
Percen- 
tile 
Rank 


Agriculture 


23 


22.9 


178.6 


58 


52 


65 


61 


53 


55 


70 


Architecture 


18 


23.6 


179.3 


60 


50 


70 


62 


61 


55 


67 


Business ' 


101 


22.1 


175.0 


50 


47 


55 


57 


50 


52 


58 


Communications 


35 


21.9 


179.3 


60 


54 


63 


67 


57 


60 


66 


Education 


15 


21.2 


175.5 


51 


50 


52 


61 


41 


58 


61 


Engineering 


142 


24.7 


181.2 


64 


59 


66 


68 


62 


63 


68 


Home Economics 


15 


21.3 


178.9 


59 


44 


73 


61 


57 


58 


66 


Liberal Arts 


136 


23.1 


178.9 


59 


55 


60 


64 


55 


60 


66 


Nursing 


16 


20.8 


175.0 


49 


^ 


55 


57 


44 


59 


50 


University 


120 


21.0 


175.7 


51 


50 


57 


57 


54 


51 


58 


Total 


621 


22.7 


178.0 


57 


53 


61 


62 


55 


56 


64 


1983 Seniors 


700 


22.3 


188.9 


* 




* 






•k 


* 



00 



i^G 



EST. GAIN 10.9 



* Percentile ranks for freshmen and seniors are not comparable because different institutions 
comprise the two norm groups on which the ranks are based. 
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SAMPLE MEANS FOR COMP 
PLOTTED ON A PERCENTILE TABLE FOR 15,581 SENIORS AT 73 INSTITUTIONS 

Key: 0=" Means for 621 FRESHMEN tested 9/83 (ACT Mean = 22»68) 
0^ Means for 700 SENIORS tested 5/83 (ACT Mean = 22.26) 
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Percen- Total Funct, in Using Using Communi- Solving Clarif- Percen- I 
tiles Score Soc. Inst. Science the Arts eating Problems Values tiles 



00- 



95- 



go- 
es- 



80- 



75- 



70— 
65— 
BO- 
SS- 



50-- 

?^ 
40—1- 

35- 

25- 

20 

5- 



209 
205 

20 1 

199 
197 




19 5 



192 



73 
71 
69 
6 8 

66 



1 

182 
- 18 0 



172 
1 6 8 
163 

- 155 

1^7 

51 



-9-0- 



•t- 73 

71 

70 
69 

6 8 



6 3 
62 



6 0 



58 



56 



Mean 

45% Men 
55% Women 



185.5 
16.5 



55 


- 51* 


52 


- 52 




1*9 


1 I* 


22 


6 3 . t« 


6 3 > 1 


6 ..3 


6 J 6 



67 
66 



62 



.1 6 1 



62 




60 
59 



58 

57 
56 



-9-9- 
69 



67 
66 



65 
'63' 



60 




59 



58 



57 



56 
55 



51* 

52 
50 

I* 7 

h 3 
1 2 



63 

6 1 

6 0 
59 

58 

"57' 



56 



79 



1+9 



k 7 



k 5 



k 1 



38 
1 5 



8 6 

Qk 

82 

81 
80 



.21- 



70 
69 
68 

66 

4- 6 2 

58 
1 k 



59 . 2 
6 , 6 



53-0 
6 * 7 



75*6 
7.2 



4 6 5 

6U 
63 

62 




6 1 
60 



57 



56 



55 



• - 5«* 



52 

5 1 
50 

4- ^* 9 



k 6 
1 0 



-100 ^ 

-95 * 
SI 

-90 
-85 

-80 i 



■75 
-70 



-4 



-60 
-65 
••50 
-45 

— Mo 

-35 
-30 
•2b 
-20 
-15 
-10 



■65 -| 



56,9 
5*8 



Description of the 15,581 Seniors in the 1983 Reference Group. 



Area of Interest 
A8Z Social Sciences 
O Natural Sciences 

l^jjj.^ Arts/Humanities 



Age Range 

IZ Age 19 or below 
59% Age 20-22 

30% Age 23-30 ^C)© 
10% Over 30 

ACT Composite (or equivalent, 
for SAT Total) scores available 207. 22-2A 
for 936A Seniors. Mean 21.0 



ACT Composite Range 
2% below 10 18% 25-27 
6% 10-12 
10% 13-15 

14% 16-18 
18% 19-21 



iO% 28-30 
2% over 30 
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<» 

o 
u 

CO 



29 



2A 



c 

u 
u 



100 

9S 



90- 



O 1 


ob- 


18 


80- 


16 




14 


/O- 


13 


6S- 


11 


60- 


10 


55" 


8 




6 
5 


40- 


3 


35- 


1 

0 

-1 


30- 


-3 


20- 


-5 


15- 


-8 


10- 


-13 


5- 
0- 



Table II 

Estimated COMP Total Score Gains for Seniors 
(Concordance Table for estimating score gains based on 7199 entering 
at 39 institutions; Percentile Table based on gains for 9221 seniors 
65 institutions.) 

Means Gains for 
67 Senior 
Institutions'^ 




9 



Estimated mean gain based 
on cross-sectional stud^r 
of 621 Fr Sl 700 Sr tested 
"if -in 1983 

Estimated mean gain based 

"t" on 382 Seniors with ACT 
=scores 



0 




I 

-2! 

4 



-11 



*See reverse side 
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67 SENIOR INSTITUTIONS FOR vvHOM MEAN SCORE GAINS 
ARE PLOTTED ON REVERSE SIDE 



Allentown College 

Andrews University 

Augustana College 

Austin Peay State University 

Baylor University 

Bethel College 

Bryan College 

Capital University 

Central University 

Clarion State College 

College of Notre Dame of Maryland 

Davis & Elkins College 

Doane College 

East Tennessee State University 
Eastern Kentucky University 
Freed-Hardeman College 
Friends University 
Glassboro State University 
Illinois Wesleyan University 
Incarnate Word College 
Iowa Wesleyan College 
Le >foyne College 
Lincoln University 
Louisiana State University 
Loyola University of Chicago 
Loyola University of New Orleans 
Marion College 

Marymount College of Virginia 
Maryville State College 
Mary^^ood College 
Memphis State University 
Michigan State University 
Middle Tennessee State University 
btomingside College 



Mount Marty College 

Mount St» Mary's College 

Nebraska Wesleyan University 

Northeast Missouri State University 

Ohio University 

Ouachita Baptist University 

Our Lady of the Lake University 

Pennsylvania State University 

Rochester Institute of Technology 

Rockhurst College 

St. Ambrose College 

St. Mary College (Kansas) 

Seattle University 

State University of New York at Plattsburgh 

Tennessee State University 

Tennessee Technological University 

Texas Christian University 

The King's College (New York) 

University of Illinois 

University of Iowa 

University of Northern Colorado 

University of Puget Sound 

University of Rhoad Island 

University of Tennessee - Chattanooga 

University of Tennessee - Knoxville 

University of Tennessee - Martin 

Upsala College 

Valley City State College 

Westraont College 

William Jewell College 

William Penn College 

William Paterson College 

York College of Pennsylvania 



CQMP Scales 



Communicating: Can send and receive information in a variety of 
modes (written, graphic, oral, numeric, and symbolic), within a 
variety of settings (one-to-one, in small and large groups), and for 
a variety of purposes (for example, to inform, to understand, to 
persuade, and to analyse). ^ 

Solving Problems: C^an analyze a variety of problems (for exam- 
ple, scientific, social, personal); select or create solutions to 
problems; and implement solutions. 

Clarifying Values: Can identify one's personal values and the 
personal values of other individuals; undersund how personal 
values develop; and analyze the implications of decisions made on 
the basis of personally held values. 

Functioning within Social Institutions: Can identify those activities 
and institutions which constitute the social aspects of a culture (for 
example, governmental and economic systems, religion, marital 
and familial institutions, employment, and civic volunteer and 
recreational organizations); understand the impact that social 
institutions have on individuals in a culture; and analyze one's own 
and others' personal functioning within social institutions. . 
Using Science and Technology: Can identify those activities and 
products which constitute the scientific/ technological aspects of a 
culture (for e.xamplc, transportation, housing, energy, processed 
food, clothing, health maintenance, entertainment and recreation, 
mood-altering, national defense, communication, and data pro- 
cessing); understand the impact of such activities and products on 
the individuals and the physical environment in a culture; and 
analyze the uses of technological products in a culture and one's 
personal use of such products. 

Using the .\rts: Can identify those activities and products which 
constitute the artistic aspects of a culture (for example, graphic art, 
music, drama, literature, dance, sculpture, film, architecture); 
understand the impact that art, in its various forms, has on 
Individuals in a culture; and analyze uses of works of art within a 
culture and one's personal use of art. 
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Response of Freshmen to Items Added to 
COMP Answer Sheet 

D In deciding to go to college, which of the following reasons was most 
important to you? (Mark only one response^) - 

Response 
Percentage 

56 To prepare myself for getting a better job 

21 To prepare myself for graduate or professional school 
8 To learn more about things that interest me 

7 To enable me to make more money 

6 To gain a general education and an appreciation^ of a wide range of 
ideas 

1 To meet new and interesting people 
.6 To increase my appreciation of culture 
.6 My parents wanted me to go 
.3 I wanted to get away from home 

0 To improve my reading and study skills 

E Where do you plan to live during your first quarter at OTK? 
78% - Dormitory 20% - At home 2% - Apartment 

F During the Fall Quarter, about how many hours per waek do you plan 
to spend working for wages? 

69% - I do not plan to work 18% - 10 to 19 hours/week .6% - 30 or more 
7% - 1 to 9 hours/week 5% - 20 to 29 hours/week hrs»/\^» 

G In deciding to attend OTK, which of the following reasons was most 
important to you? (Mark only one respon.'5e.) 

34 OTK offers a strong program in my field of interest 

22 OTK is close to my home 
13 UTK has a very good academic reputation 

8 Someone v^o had attended OTK advised me to come here 
6 My relatives wanted me to come to OTK 
5 I was offered financial assistance 
4 A friend suggested I come to OTK 
4 OTK has low tuition 

2 A representative of OTK recruited me 
1 My guidance counselor advised me to come to OTK 

121 Are you currently classified as an in-state or out-of-state student 
at OTK? 

86% - In-state 14% - Out-of-state 
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What is your marital status? 

99% - Single .5% - Married ,2% - Divorced 



In which type of community did you receive most of your education 
prior to college? 

8 Rural area or small town (town less than 2500) 

34 Town of 2500 to 25,000 

28 City of 25,000 to 100,000 

23 Suburbs of city over 100,000 

7 Inner-city (city over 100,000) 



What was the highest level of education attained by your mother? 

7 Did not graduate from high school 

30 High school graduate 

31 Beyond high school but did not complete 4 years of college 

20 Graduate of four-year college 

12 Attended graduate or professional school 

What was the highest level of education attained by your father? 

8 Did not graduate from high school 
19 High school graduate 

21 Beyond high school but did not complete 4 years of college 
30 Graduate of a four-year college 

23 Attended graduate or professional school 

Before deciding to come to UTK you probably received written infor- 
mation from a number of colleges and universities which described 
their academic programs and student services. Compared with the 
information you received from other colleges, was that which you 
received from OTK 

22 More helpful? 

60 About as helpful? 

8 Not as helpful? 
10 I received no information from UTK? 



Students hear many things about the reputation of a university before 
they enroll. Which of the following aspects of the OTK reputation 
was most attractive to you as you made your decision to enroll here? 
(Mark only one response.) 

51 Quality of academic programs 

19 Quality of social life 

8 Quality of cultural and recreational activities for students 

4 Quality of the athletics program 

18 None of the above 
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ISSUES FOR DISCUSSION IN FY 85 BUDGET HEARINGS 
University of Tennessee, Knoxville 

I. Describe the mission of the unit and its relationship to the 
UTK mission. 

II. List the goals and supporting objectives which the unit has 
established to carry out its mission and identify additional 
resources required to achieve the objectives. (Section XI 
below and the attached form provide for the detailed reporting 
of new resource requirements.) 

III. Can the un.^t use enrollment reductions to improve quality 
and/or achieve other goals? 

IV. If the unit has had an academic program review within the last 
five years r discuss the recommendations which have been 
implemented and those which remain to be implemented r noting 
any barriers to implementation. 

V. What does the unit consider to be its greatest strengths? Its 
major weaknesses? 

VI. What potential exists for the unit to achieve (or maintain) 

nationad or regional prominence? Discuss the related resource 
requirements and the criteria used to eiscertain the relative 
position of the unit. (Such factors may include, but are not 
limited to, quality of students; quality of faculty; quality and 
volume of research; and program quality as indicated, by assess- 
ments of student outcomes, national rankings, academic program 
reviews, and other peer assessments*) 

VII* Specify the unit's potentiad uses of research incentive funds. 
What is the estimated ratio of dollars returned to each dollar 
invested? What qualitative improvements in research could be 
underwritten through an investment of research incentive 
funds? 

VIII. Vrtiat is the perceived need within the unit for central funding 
for faculty development opportunities? 

IX. What are the unit's highest priorities for improvements in 
facilities? 

X. If the unit is presently charging laboratory feeSf indicate the 
level of fees charged and the basis for the charge. If the 
unit is not presently charging a lab fee and has a rationale 
for proposing the implementation of such a fee, indicate the 
proposed level and rationale for the fee. 

XI. Using the attached form, provide the requested information for 
the funding adjustments required during the next three years to 
accomplish the goals and objectives of the unit. 
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